gms | German Medical Science

Studies utilizing survey questionnaires to collect epidemiologic data rely on the accuracy of participants’ self-reporting. An important aspect of survey research is assessing the quality of this data. For information associated with the use of health services, validity (accuracy) can be measured by comparing self-reported information with medical records. Another aspect addressing potential measurement errors in epidemiological studies is the reliability (consistency) of self-reported data. The test-retest approach, in which study participants are re-interviewed and the level of agreement between questionnaires is measured, is a common method for assessing reliability.

As part of the quality-control protocol for a large population-based case-control study of the association between hormone replacement therapy (HRT) and breast cancer in postmenopausal women in Germany, we used test-retest to evaluate the reliability of women’s self-reporting of a number of health-related factors, including demographics, anthropometrics, reproductive history, family history, lifestyle, and use of HRT. We administered a shortened version of the original study questionnaire to a randomly selected sub-sample of 62 cases and 61 controls (dichotomized at 60 years) from participants recruited in the first year of the three-year study. Agreement between the two interviews was assessed using Cohen’s Kappa (К), the Intraclass Correlation Coefficient (ICC), and proportion of agreement. A separate quality-control study including the current sample examined validity by checking women’s self-reported HRT use against data from physician reports.

Mean duration between interviews was 10 months. Participants were interviewed and re-interviewed by the same interviewer. There was very good overall agreement between the two questionnaires for HRT ever/never use (К=0.90), type of therapy (mono/combination) (first phase of use: К=0.75, last phase of use: К=0.87), and form of application (first phase of use: 86% agreement, last phase of use: 95% agreement). Agreement was also high for demographic/anthropometric variables (agreement range: 87-100%), reproductive factors (agreement range: 70-100%), past alcohol consumption (ICC: 0.64), and past smoking duration (ICC: 0.68), as well as past physical activity (duration exact agreement: 44%, ± 1 hour agreement: 81%). An examination of duration of HRT use revealed that disagreement led to misclassification of duration group (≤ 5 years HRT use vs. > 5 years HRT use) among only 20% of participants for the first phase of HRT use and 10% of participants for the last phase of HRT use. Cases showed slightly better agreement for the first phase of HRT use than did controls (86% and 71% agreement, respectively), but this difference was not statistically significant. No differential agreement by disease status was seen for the last phase of HRT use, and we observed no differential agreement of HRT duration by age group.

Small differential agreement by disease status for HRT duration may indicate that controls are less motivated than cases to recall with precision HRT duration of use in the distant past in a repeated survey. A validity study on the same sample found good agreement between self-reported duration of lifetime HRT use and prescription data, with no significant difference between cases and controls. Overall, these combined findings suggest that self-reported data gathered in the context of a study on breast cancer and HRT are highly repeatable, implying good quality and low likelihood of misclassification.