gms | German Medical Science

To make informed health decisions, patients need the relevant information presented in an adequate manner. Criteria for high quality patient information have been established based on ethical and legal and scientific considerations [1]. According to these criteria patient information has to be understandable, evidence based, complete, relevant for patients, and balanced. Developers are guided in selecting content and appropriate presentation methods. This has been referred to as “evidence-based patient information (EBPI)” [2]. However, successful communication of probabilistic information, such as health risks, frequencies of benefit and harm of therapies, prognostic or diagnostic tests still seems the most difficult task in patient information and even physician information [3], [4]. In particular, single event probabilities, conditional probabilities, and relative risks have been found to be confusing, largely due to poor presentation of information [3]. Erroneous communication of risks may cause serious consequences such as medical decisions based on inappropriate expectations. Various graphical formats exist to visualise empirical information to patients or consumers including bar graphs, survival curves, and pie charts [1], [5], [6]. In particular, diagrams comprising different forms of multiple figures (e.g. stick-figures or smileys) are frequently used within EBPI or patient decision aids [7] to visualize disease risks, and benefits and harms of medical interventions. These illustrations often show multi-figure pictographs (MFPs), i.e. an arrangement of figures (N=100 or N=1000) representing the distribution of patients with effects/no effects or side effects/no side effects of an intervention. Supplemental graphical illustration of proportions of events of benefit or harm has been shown to support understanding when compared to text only risk information [8] and to positively influence physician patient communication [4]. Within the various graphical formats MFPs have been proven advantageous [9], [10], [11]. Qualitative data suggest that people prefer MFPs to bar charts since it “is clearer that you’re talking about human beings and not statistics” [12]. However, it is not clear, how such risk formats are best composed to support understanding and to improve decision making quality. Evaluation of pictographs’ ability to appropriately mediate risk information needs to consider two aspects of such information: Firstly, the information aims at accurately transmitting a specific proportion e.g. the percentage of people who benefit from a certain therapy and of people who do not benefit. Secondly, the information has to indicate its probabilistic character, i.e. that uncertainty remains about the prediction whether an individual belongs to the group with or without a benefit.

The process of information is an interaction of the graphical presentation with the perceptual, emotional, and cognitive structure of its recipients. For instance, as a result from emotional involvement with the decision, patients might tend to irrationally process given quantities. This may lead to over- or underestimation or over-salient perception of events that objectively are not prominently presented. Also, depending on individual degrees of tolerance towards complex and ambiguous situations – called uncertainty tolerance [13] – patients’ information processing might vary widely. As a cognitive style, uncertainty tolerance has been proven an important factor in learning and information processing not only in education but also in health communication [13]. Graphical formats to communicate likelihoods of health related events should as far as possible be robust against such biases. Therefore, understanding of internal mechanisms of processing risk information might be helpful to minimize cognitive bias.

To adequately account for the state of knowledge in this area we systematically searched Medline, EMBASE, PsycINFO, PSYNDEXplus, and CINAHL (last search June 2010) for original studies investigating the MFP-method used to visualize probabilistic information in the context of shared decision making. The search strategy was based on the latest Cochrane Review on decision aids [14]. Searches were limited to English or German language publications but not limited according to type of study design or publication year. Only original studies were included that compared MFPs with other graphical formats to communicate likelihoods in the context of shared decision making or that compared different design features of MFPs. Reference lists of retrieved publications were screened for additional publications. Retrieved material was analysed independently by two investigators (JK and ML).

After duplicates were removed a total of 1129 potentially relevant titles were identified. 1111 of which could be excluded with certainty. Of the remaining 18 references assessed for eligibility as full-text, four were found relevant for the study question [12], [15], [16], [17].

Two studies compared MFP with other presentation formats [16], [17].

In a randomised trial [16] response time of a hypothetical medical choice and accuracy of recall of previously presented risks were compared among six different presentation formats (pie charts, vertical bars, horizontal bars, numbers, consecutively arranged MFPs, and randomly arranged MFPs).

Pie charts and random ovals caused the slowest and less accurate performances [16].

In another study [17] focus groups were used to assess the ease of interpretation, simplicity and perceived vulnerability against bias of perceived risk magnitude in bar graphs compared to MFPs. MFPs turned out to be appreciated for higher salience compared to bar graphs, but also to be associated with potential bias regarding the accurate perception of magnitude of a risk [17].

Four studies evaluated differential effects of a variety of features of MFPs [12], [15], [16], [17]. In a controlled study, differences in accuracy and speed of perception of quantitative risk information were surveyed for specific types of MFPs, in particular two single pictographs compared to one compound pictograph, pictographs arranged in horizontal compared to vertical direction, and shading compared to no shading of the highlighted groups of figures in the pictographs [15]. Horizontal pictographs were perceived faster and more accurately than vertically arranged ones and two single pictographs showed faster perception compared to compound formats particularly with shaded background. Although shading increased response time in compound pictograms, this feature was preferred to the more effective forms [15].

In another study [12] qualitative methods were used to investigate the effects of 1) an interactive application of MFPs in comparison to a static application and 2) random pictograms compared to consecutive arrangements. Animation and interactive use of MFPs evoked more emotional response, both positive and negative than viewing static graphs. Random arrangements of figures (indicating persons affected by the disease throughout a group of figures) made it more difficult to judge the proportions but were considered more realistic than consecutive arrangements [12].

Two variants of pictographs were part of an aforementioned trial [15] that compared formats regarding perception speed and accuracy. Consecutively arranged pictographs were superior to randomly arranged formats regarding both accuracy and speed of perception.

One study employed focus groups with women who were asked to comment different formats illustrating breast cancer risk [17]. The study investigated the size of the denominator MFPs are based on (amount of individuals in a reference group). MFPs using a lower denominator were preferred to those with higher denominator. In addition, the order (arrangement) of pictograms was assessed. Random order was difficult to process for some participants, however, also recognized as part of concept of conveying random occurrence.

In summary, it seems that formats subjectively perceived best for making a choice to some extent differ from those best for estimating the size of an amount.

The actual state of evidence on MFPs does not provide much guidance whether or how to use this format to support understanding of probabilistic risk information. Whereas evidence suggests that consecutive arrangements and numerical data are perceived easier and more accurately, potential advantages of the random order as found in some qualitative studies are hitherto not sufficiently understood. No studies were identified, evaluating different types of arrangements regarding potential differences in accuracy of information recall and the understanding of randomness.

Having in mind the importance of patients’ accurate comprehension of numerical risk data and the issue of randomness in the concept of risk, the need becomes obvious to understand the effects of pictogram arrangement on cognitive and emotional perception of risk.

This study aimed at investigating advantages of the use of 100 figure pictographs presented in unsorted compared to those in consecutive order. Although the common way to present pictographs in consecutive order seems more likely to yield precise comprehension of proportions, unsorted pictograms are recognized closer to reality. Perception of this information might therefore be less vulnerable for irrational biases evoked by individual cognitive attribution styles or uncertainty tolerance. We assessed the effects of the order within a pictograph on accuracy of comprehension and on vulnerability for irrational biases.

A randomised controlled trial was conducted in 2007 testing two experimental conditions: patient information using 100 figure pictographs 1) in unsorted (UP group) and 2) in consecutive arrangement (CP group). Within each condition two hypothetical scenarios representing different levels of relevance were provided in order to investigate potential differences in information processing caused by varying degrees of importance of the content.

The study population consisted of a convenience sample of adults suffering from multiple sclerosis (MS). Participants were recruited at the MS Outpatient Unit of University Medical Centre Hamburg-Eppendorf, the Centre of Neurology Hamburg-Bramfeld, and the Hamburg MS Society. Patients with confirmed MS were eligible if they were between 18 and 70 years old, German speaking and not obviously cognitively or, according to own statement, heavily visually impaired.

The study experiment was framed as a booklet for patients providing information on methods used in EBPI to communicate risks and benefits. Study participants were asked to read the booklet thoroughly and then to recall specific parts of it. The booklet consisted of 17 charts. (The experiment booklets are attached online to this paper: consecutive order in English language (Attachment 1 [Attach. 1]), consecutive order in German language (Attachment 2 [Attach. 2]), random order in English language (Attachment 3 [Attach. 3]), random order in German language (Attachment 4 [Attach. 4])). The first five charts introduced the issue of absolute and relative risk reduction. Two examples were given by using pictures of glasses each containing 1000 blue and yellow pearls in different quantities referring to proportions of persons with benefit or no-benefit. This introductory part was intended to motivate participants and to equalize potentially varying degrees of understanding about the concept of risk. The remaining 12 charts contained the actual experiment, risk information, embedded in two hypothetical scenarios presenting treatment decisions. Each consisted of two “therapeutic” options for which probabilities of benefit and side effects were displayed. The first scenario – ‘garden pond algae pollution’ – illustrated a non medical problem and was assumed to be of low relevance and is subsequently referred to as “neutral scenario (NSc)”. As ‘therapeutic’ option we chose the ‘utilization of water slugs’ compared to ‘no action’. The second scenario – ‘hypothetical MS-treatment’ – was assumed to be perceived as highly relevant for participants (‘relevant scenario’ (RSc)). In both scenarios, fictitious probabilities of benefits and side effects were given. Proportions were based on existing MS-treatment effects. To avoid recognition, probabilities were slightly varied between both scenarios. A previously evaluated five step proceeding was used to structure the two scenarios. This approach was shown to be feasible to provide risk information about specific treatment effects comprehensibly [18]: 1) the four possible outcomes were explained theoretically: intervention & no benefit, intervention & benefit, no intervention/placebo & no benefit, no intervention/placebo & benefit (2 charts). 2) Quantities of ponds/patients showing no benefit and those showing benefit were presented in two 100 figure pictographs, one for treatment and one for placebo/no treatment (NSc: without treatment 31, with treatment 45 ponds/patients showing improvement out of 100; RSc: without treatment ponds/patients showing no worsening 31, with treatment 43 ponds/patients showing no worsening) (1 chart). 3) One chart displayed the absolute risk reduction using one single compound pictograph (NSc: 14 out of 100; RSc: 12 out of 100) (1 chart). 4) The potential side effects were explained theoretically: having (or not having) side effects with (or without) intervention (2 charts). 5) An additional chart visualized the quantity of side effects using another 100 figure pictograph.

Pictographs were presented without any numerical or textual explanation (Figure 1 [Fig. 1]), so that participants could gather the quantitative information solely from the chart. This proceeding intended to maximize potential differences between the two formats.

Recall of quantities was assessed for benefit and side effects for NSc and RSc. Accuracy of recall of benefit regarding NSc was defined as the primary outcome. After participants had thoroughly studied the charts, they were interviewed by a trained researcher. The interview was structured using 10 additional charts.

Participants had to identify the correct information and to translate the graphically displayed proportions into frequency values (“how many are affected out of 100?”). To prevent participants from perceiving the situation as a performance test, the experiment was framed as an information session. The participants had to complete both scenarios (NSc & RSc) under the same conditions before they were interviewed. Recall performance was assumed not only being a matter of (sensory) perception but also a matter of cognitive processing such as memory and subjective appraisal. The latter is sensitive to any irrational assumptions, attitudes or styles of evaluation. As potentially relevant to this kind of process, self-serving bias might impact on the result of cognitive information processing [19]. The self-serving bias refers to a tendency to evaluate ambiguous information in a way that is beneficial for one’s interests. Here, self-serving bias would lead to overestimation of the own chance of benefit and underestimation of the risk to be affected by side effects accordingly. However, we assumed such kind of biases only in the processing of emotionally important contents i.e. the RSc. Therefore, intra-individual differences in recall performance between NSc and RSc were assumed to indicate a presentation form’s vulnerability to evoke such cognitive bias. This was defined as the secondary outcome. As another indicator of patients’ tendency to irrationally bias their cognitive processing either as an optimism bias or as a pessimism bias [20], disease specific risk attribution was assessed by asking participants to estimate their personal risk of becoming wheelchair bound (according to [21]). Using visual analogue scales (VAS), patients had to rate their risk higher or lower compared to a fictive person at equal age and state of disease, which was represented in the middle of the VAS. This approach had been used before in a study evaluating risk perception of MS patients [21]. Estimating one’s own risk higher than that of a matching person indicates a problem-sensitive and pessimistic coping style. Downsizing one’s own risk indicates a more repressive coping style. Different VAS were presented separately referring to the 2-year, 10-year and lifetime risk. In addition, by using the 8 item uncertainty tolerance-questionnaire [13], degrees of participants’ cognitive affinity to uncertain and complex situations were assessed.

To determine whether presentation forms had an impact on choices, a hypothetical choice was assessed. To avoid responses according to patients' own situation, participants were asked “Would you recommend this treatment to another person affected by MS”. Participants could respond on a 4 point Likert scale ranging from 0 (absolutely yes), 1 (tend to ‘yes’), 2 (tend to ‘no’), and 3 (absolutely no) for each scenario. Furthermore, we qualitatively assessed reasons for reported intentions by an open format question. To control for possible confounding with recall accuracy, familiarity with the specific scenario was assessed. Using three point Likert scales, participants were asked for instance whether they were personally concerned by ‘algae pollution of garden ponds’, whether they were experienced in making MS-treatment decisions, and whether they were familiar with MFPs similar to those used in this study. To describe the study sample demographically and regarding disease related aspects, age, gender, highest level of education, duration of MS in months, and the Cambridge Multiple Sclerosis Basic Score (CAMBS) [22] were assessed. The CAMBS considers four aspects of MS: disability and impairment, relapse, progression, and handicap. The whole study experiment was piloted with ten MS patients to warrant readability and to achieve indicators for sample size calculation.

Allocation to presentation of either UP or CP was conducted randomly and concealed by drawing pre-coded questionnaires from a batch after a participant had given informed consent. Subsequently, baseline data including uncertainty tolerance and risk attribution were surveyed. In face-to-face interviews participants were asked to step-wise work through all pages of the booklet. If required, the interviewer gave assistance. To avoid misunderstanding of the concepts of relative risk reduction and absolute risk reduction, the investigator checked comprehension when participants reached the chart explaining those concepts, using a standardized question. If necessary, additional explanation was given.

After participants had read the experimental information recall of presented frequencies and the hypothetical choice were assessed. Finally, in order to assess the participants’ preferences regarding presentation formats, alternative formats were presented together with the question on preference.

1.

We expected frequencies from the CP to be recalled more accurately.

2.

On the other hand however, we expected the CP to promote a cognitive-motivational bias of the information process. We assumed an easier identification of figures representing individuals with a positive outcome to facilitate self-serving bias, whereas the random arrangement might strengthen patients’ awareness of the probabilistic nature of risk. Driven by individual attribution styles, this motivational bias was presumed to particularly affect the processing of the RSc, i.e. we expected (hypothetical) choices to follow the perception bias.

3.

Participants tending to overestimate benefits and to underestimate harms were assumed to make more positive choices and vice versa.

4.

Accordingly, we expected risk attribution (RA) to influence processing of risk information. Thus, optimistic RA was assumed to be associated with overestimation of treatment effects and underestimation of side effects.

5.

We expected participants to prefer CPs compared to UPs. However, this preference was expected to decrease with increasing uncertainty tolerance.

Accuracy of recall was drawn from the NSc data by calculating differences between raw values (frequency of benefit and side effects) and the corresponding true value. If the given value in the booklet was 15% risk, a corresponding recall risk of 25% meant a 10% error. Cognitive bias was defined on an intra-individual level based on the additional recall error occurring in the RSc in relation to the NSc. Cognitive bias was operationalized by calculating the difference of errors made when recalling the frequencies of the RSc and of the NSc, expressed as absolute value on a 0 to 100 scale. Experimental conditions were compared regarding accuracy of recall and cognitive bias using unpaired t-tests.

Participants rating their personal risk of becoming wheelchair bound at least 10% higher or lower compared to their matches were defined as vulnerable towards cognitive bias. In the subgroups of these participants showing either a pronounced optimistic or pessimistic attitude, the secondary endpoint (cognitive bias) was tested between the experimental conditions. Using Pearson correlation coefficients, risk attribution and uncertainty tolerance were correlated to accuracy regarding each domain of recall: frequencies of benefit and side effects for NSc and RSc. Transformed into a scale ranging from minus 1.5 to plus 1.5, data on choice were correlated to accuracy as well.

The primary endpoint was the accuracy of recall of benefit in the NSc, where effects of order of pictographs were assumed to be independent of cognitive biases. Sample size calculation was based on data from the pilot study (N=10) (G-power 3.0 [23]). Assuming 90% power, 0.05 α-level (two sided) and a 16.5 standard deviation, a minimal total sample size of N=96 was calculated, required to identify a mean difference of 10% in the primary endpoint, which we considered important.

A total of 111 patients were assigned to the two study conditions (Figure 2 [Fig. 2]). Demographic and disease-related variables were equally distributed with respect to gender, level of education, disease duration, and disability. In the UP-group participants were older (Table 1 [Tab. 1]).

Accuracy of recall of frequencies was low in both groups (mean of absolute error values for benefit_NSc=16.5%, side effects_NSc=11.4%, benefit_RSc=18.5%, side effect_RSc=13.2%). Patients overestimated benefits more than 100% (mean error for the NSc=+15.0, SD=18.7, for RSc=+17.7, SD=19.0). Recall of frequencies related to side effects were more precise, however with pronounced variance (mean error for NSc=–4.9, SD=13.1; RSc=+2.7, SD=17.2). While study groups did not differ in their accuracy of recalling benefits, the recall of side effects was more accurate in the CP-group (Table 2 [Tab. 2]). Contradicting our hypothesis, the cognitive bias when recalling benefits was higher in the UP-group than in the CP-group and equal for side effects in both groups (Table 2 [Tab. 2]). Results were similar in subgroup analyses (Table 2 [Tab. 2]) that included only participants who showed a considerable tendency to make irrational personal risk attributions. Accordingly, risk attribution was not correlated to accuracy in three of the four domains of recall (NSc_benefit r =–0.027, NSc_{side effects} r=.147, RSc_benefit r=0.118, RSc _{side effects} r=0.204, Pearson correlations with comparative two year risk of becoming wheelchair bound) and slightly associated with RC_{side effect}. Participants with a more pessimistic attribution style tended to overestimate side effects and vice versa (r=0.21, p=.035). Uncertainty tolerance was not correlated with the accuracy in all four recall domains. However, a low negative correlation appeared, indicating that higher uncertainty tolerance might be associated with lower irrational information processing (uncertainty tolerance and cognitive bias_benefit r=–0.20, p=0.04; uncertainty tolerance cognitive bias_{side effects} r=–0.13, p=0.18). Hypothetical choices were predominantly positive towards ‘treatment’ in both groups and in both scenarios with a tendency to more positive choices in the UP-group (NC: p=0.22, RC: p=0.09). Accuracy was not correlated with choice in the total sample, indicating that differences in perceiving the information had no impact on the decision. Unsorted pictographs were preferred by only two participants (2%), one out of each study group. Confounding variables were equally distributed between both conditions, except of familiarity with pictograms (UP-group: 75%=“not at all”, CP-group=62%). Subgroup analyses 1) of participants not concerned with garden pond pollution (Table 2 [Tab. 2]), 2) of participants without previous experience with MS-treatment decisions (Table 2 [Tab. 2]), and 3) of participants not familiar with using pictographs did not lead to different results (Table 2 [Tab. 2]).

The study demonstrated that participants had pronounced difficulties to correctly perceive and process risk information presented by using MFPs. Recall of benefits displayed showed marked overestimation. This result was not explained by either experimental variation of pictograph format (unsorted vs. consecutive) or by indicators of irrational cognitive processing (risk attribution and uncertainty tolerance). We found no clear difference in patients’ ability to process consecutive or unsorted arrangements. Although patients showed better recall of side effects when using consecutive pictographs, no corresponding effect was seen for presentation of benefit. In contradiction to our hypothesis that consecutive arrangement could mislead vulnerable individuals to bias their perception in accordance with their risk attribution style, we found a more pronounced cognitive bias in the UP-group for benefit, while no such effect was seen for side effects. Also, no correlation was seen between accuracy of recall, risk attribution and uncertainty tolerance.

Regarding the ambiguous results, recommendations about how to present MFPs seem premature. Moreover, generalizability of the study results may be limited by the experimental character of the design. The unusual presentation of frequencies (figuratively and without supplemental numerical or textual information) might have increased uncertainty of recall and might even have clouded actually existing effects [12]. The present study has further limitations. The presentation example used only one certain proportion which was chosen as representative for MS treatment effects. It cannot be ruled out that using different proportions would yield different results. Furthermore, given proportions did not allow for recall mistakes up to the same extent in both directions. Therefore, overestimation was more likely than underestimation. However, systematic variation of proportions would have further increased the required sample size. Furthermore, the hypothetical character of the given information may have limited the validity of reported choices. It is questionable whether and to which extent reported choices were based on the given information or based on already existing beliefs. In contrast to our study Schapira [17], Ancker [12], and Feldman-Stewart [16] found that randomly arranged pictographs were more difficult to process cognitively. Interestingly, although participants expressed a clear preference for the consecutive order, our study does not indicate that comprehension is worse in the unsorted format. This preference might be explained by the higher cognitive efforts required to process unsorted arrangements. Taking into account the indistinct results, further research seems warranted e.g. with modified design features. Also, it could be helpful to address bigger samples of healthy persons not previously contaminated with either pictograph presentation or medical decisions.

There are a number of inherent goals of presenting different frequency formats to support patients’ decision making [8]. Beyond accuracy and speed of capturing given frequencies, other aspects of the information process should be considered. On the cognitive-behavioural level usability, preference or behaviour planning might be important parameters. Although potentially interfering with some of the abovementioned goals, addressing uncertainty of given risk information can be another important goal. Accordingly, research should focus on awareness of randomness as another potential effect of presentation of frequency formats. Developers should use different existing formats according to their priority of specific communication goals [17][. For instance, some evidence exists that unsorted arrangements of figures emphasize randomness [12], [17] while consecutive arrangements seem to emphasize accuracy of perception [12], [15], [17]. Therefore, developers could purposefully combine these formats to facilitate either accurate perception or awareness of randomness or both.

Evidence based patient information remains challenging as tools and strategies are still far from reaching satisfactory standards. Therefore, further research about optimizing technologies to communicate probabilities is required.

CP-group = study group receiving pictograms arranged consecutively

EBPI = Evidence Based Patient Information

MFP = multi-figure pictograph

MS = multiple sclerosis

NSc = neutral scenario

RSc = highly relevant scenario

UP-group = unsorted pictogram group

VAS = visual analogue scales

RA = risk attribution

We thank Petra Gutzeit and Johanna Kaiser for their support in designing the study protocol and data assessment.

Christoph Heesen has received financial support from Biogen Idec, Bayer-Health Care, Serono, Sanfor-Aventis and Teva Pharma; Sascha Köpke, Jürgen Kasper, Ingrid Mühlhauser and Matthias Lenz have nothing to declare. We are not aware of any potential conflict of interests.