gms | German Medical Science

GMS Psycho-Social-Medicine

Corporate journal of German scientific societies in psychosocial medicine

ISSN 1860-5214

Screening for Generalized Anxiety Disorder in inpatient psychosomatic rehabilitation: pathological worry and the impact of depressive symptoms

Screening nach der Generalisierten Angststörung in einer Fachklinik für Psychotherapie und Psychosomatik: pathologisches Sich-Sorgen und der Einfluss von Depressivität

Research Article

  • corresponding author Simone Salzer - Clinic of Psychosomatic Medicine and Psychotherapy, Georg-August University Goettingen, Germany
  • Christian Stiller - Clinic of Psychosomatic Medicine and Psychotherapy, Georg-August University Goettingen, Germany
  • Achim Tacke-Pook - Paracelsus-Roswitha-Klinik, Bad Gandersheim, Germany
  • Claus Jacobi - Paracelsus-Roswitha-Klinik, Bad Gandersheim, Germany
  • Eric Leibing - Clinic of Psychosomatic Medicine and Psychotherapy, Georg-August University Goettingen, Germany

GMS Psychosoc Med 2009;6:Doc02

DOI: 10.3205/psm000058, URN: urn:nbn:de:0183-psm0000580

Published: July 9, 2009

© 2009 Salzer et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by-nc-nd/3.0/deed.en). You are free: to Share – to copy, distribute and transmit the work, provided the original author and source are credited.


Abstract

Objective: Pathological worry is considered to be a defining feature for Generalized Anxiety Disorder (GAD). The Penn State Worry Questionnaire (PSWQ) is an instrument for assessing pathological worry. Two earlier studies demonstrated the suitability of the PSWQ as screening instrument for GAD in outpatient and non-clinical samples. This study examined the suitability of the PSWQ as a screening instrument for GAD in a German inpatient sample (N=237). Furthermore, a comparison of patients with GAD and patients with depression and other anxiety disorders regarding pathological worry and depression was carried out in a sub-sample of N=118 patients.

Method: Cut-off scores optimizing sensitivity, optimizing specificity and simultaneously optimizing both sensitivity and specificity were calculated for the PSWQ score by receiver operating characteristic analysis (ROC). Differences regarding pathological worry and depression measured by the PSWQ and the Beck Depression Inventory (BDI) across five diagnostic subgroups were examined by conducting one-way ANOVAs. The influence of depression on pathological worry was controlled by conducting an ANCOVA with BDI score as a covariate.

Results: The ROC analysis showed an area under the curve of AUC=.67 (p=0.02) with only 54.4% of the patients correctly classified.

Comparison of diagnostic subgroups showed that after controlling the influence of depression, differences referring to pathological worry between diagnostic subgroups no longer existed.

Conclusions: Contrary to the earlier results we found that the use of the PSWQ as a screening instrument for GAD at least in a sample of psychotherapy inpatients is not meaningful. Instead of that, the PSWQ can be used to discriminate high from low worriers in clinical samples. Thus, the instrument can be useful in establishing e.g. symptom-oriented group interventions as they are established in behavioural-medicine inpatient settings. Furthermore, our findings stress the influence of (comorbid) depressive symptoms on the process of worrying.

Keywords: worry, Generalized Anxiety Disorder, Penn State Worry Questionnaire, depression, inpatients

Zusammenfassung

Zielsetzung: Pathologisches Sich-Sorgen gilt als eines der Kernmerkmale der Generalisierten Angststörung (GAS). Der Penn State Worry Questionnaire (PSWQ) erfasst pathologisches Sich-Sorgen. In zwei früheren Studien wurde die Eignung des PSWQ als Screening-Instrument für GAS bei ambulanten Patienten und in nicht-klinischen Stichproben nachgewiesen. Die vorliegende Studie untersuchte die Eignung des PSWQ als Screening-Instrument für GAS bei stationären Patienten einer Klinik für Psychotherapie und Psychosomatik (N=237). Darüber hinaus wurden Patienten mit GAD sowie Patienten mit Depression und anderen Angststörungen hinsichtlich des pathologischen Sich-Sorgens und der depressiven Symptomatik in einer Substichprobe von N=118 Patienten miteinander verglichen.

Methodik: Anhand einer Receiver Operating Characteristic Analysis (ROC) wurden Cut-off-Werte für den PSWQ bestimmt, zum einen mit einer Betonung der Sensitivitität, zum anderen der Spezifizität sowie einer Ausbalancierung beider. Unterschiede im pathologischen Sich-Sorgen und der depressiven Symptomatik, erfasst mit dem PSWQ und dem Beck Depressionsinventar (BDI) über fünf diagnostische Subgruppen, wurden anhand einfaktorieller Varianzanalysen untersucht. Der Einfluss der depressiven Symptomatik wurde anhand einer Kovarianzanalyse mit dem BDI als Kovariate untersucht.

Ergebnisse: Die ROC Analyse ergab ein 'area under the curve' von AUC=,67 (p=0,02) mit nur 54,4% richtig klassifizierten Patienten.

Ein Vergleich der diagnostischen Subgruppen im Hinblick auf das Sich-Sorgen zeigte, dass diese keine Unterschiede mehr aufwiesen, wenn der Einfluss der depressiven Symptomatik kontrolliert wurde.

Fazit: Entgegen früheren Befunden scheint der PSWQ als Screening-Instrument für GAS zumindest in der untersuchten Stichprobe von stationären Patienten nicht geeignet zu sein. Der PSWQ kann jedoch dazu verwendet werden, Patienten mit ausgeprägtem Sich-Sorgen zu identifizieren und so die Zuweisung z. B. zu indikativen Gruppen in der stationären verhaltensmedizinischen Versorgung zu erleichtern. Darüber hinaus betonen die Ergebnisse den Einfluss von (komorbider) Depressivität auf den Prozess des Sich-Sorgens.


Introduction

Generalized anxiety disorder (GAD) is a common disorder with high 12-month prevalence of 3.1% [1]. The disorder is associated with severe psychosocial impairment, is typically chronic in course and often associated with comorbid mental disorders such as depressive disorders and other anxiety disorders [2], [3], [4]. GAD is characterised by chronic, excessive and uncontrollable worrying referring to different areas of everyday life [5], [6]. In addition, patients with GAD suffer from complaints such as irritability, restlessness, muscle tension, difficulty concentrating, sleep difficulties, feeling keyed up and easy fatigability [7]. Pathological worry is considered to be a defining feature for GAD [8]. These worries are experienced as uncontrollable and patients spend considerable time worrying [9]. Obviously, patients suffering from anxiety disorders in general are concerned by several worries. Nevertheless, Chelminski and Zimmerman [10] found significantly higher levels of pathological worry for patients with GAD in comparison to other anxiety disorders.

In two earlier studies, the Penn State Worry Questionnaire (PSWQ) [11] was used as a screening instrument for GAD [12], [13]. The PSWQ was developed to measure the specific trait of worry in both clinical and non-clinical populations. It is a commonly used and psychometrically sound measure [14]. Individuals diagnosed with GAD score significantly higher on the PSWQ than do those who meet only some of the GAD criteria [11] as well as individuals meeting the criteria for other anxiety disorders [15].

Behar et al. [12] and Fresco et al. [13] examined the screening suitability of the PSWQ in different samples by the use of receiver operating characteristic analyses (ROC). A ROC analysis enables to evaluate the ability of a test to discriminate individuals with a characteristic (e.g., GAD) from individuals without that characteristic. Behar et al. [12] compared PSWQ scores from 159 treatment-seeking GAD patients with PSWQ scores of 113 nonanxious controls. A PSWQ cut-off-score of 45 provided the best balance of sensitivity (0.99) and specificity (0.98). In a second study, Behar et al. [12] examined a large sample of unselected college students (N=2449). In this sample, a PSWQ cut-off-score of 62 provided the best balance of sensitivity (0.86) and specificity (0.75). One limitation of this study was the assessment of GAD by self-report measures. Further, the sample did not consist of participants seeking treatment.

Fresco et al. [13] examined the ability of the PSWQ to correctly identify patients with GAD in a sample of outpatients seeking treatment at an anxiety disorders clinic. The authors investigated three diagnostic groups: patients with GAD (n=28), patients with Social Phobia (n=114) and patients with a primary diagnosis of Social Phobia and a secondary diagnosis of GAD (n=22). By discriminating 50 patients with either primary or secondary GAD from 114 patients with Social Phobia, the analysis revealed a strong ROC curve for the PSWQ total score (Area Under Curve [AUC]=0.74) that was significantly better than chance in classifying individuals with GAD versus Social Phobia. When sensitivity and specificity were both optimized (cut-off score=65), 63.4% of the patients were correctly classified. Because of a high degree of overlap between the two anxiety disorders, the authors take their results as a strong support for the use of the PSWQ as initial screening instrument for GAD. Nevertheless, Fresco et al. [13] point out that the generalization of theses findings are strongly limited due to the influence of the modalities of a study (e.g., sample, diagnostic procedure, context of exploration).

As the two earlier studies [12], [13] referred to college students and outpatient samples, our research question was whether the PSWQ can be used as a screening instrument for GAD in a German inpatient sample. (As a special feature of the health care system in Germany, not only patients with severe psychiatric disorders, but also patients with less severe disorders such as anxiety disorders, depression and somatoform disorders are treated in inpatient health care.) As the discrepancy between the high frequency of GAD, and the duration and validity of its diagnosis [16] shows, the detection rates regarding this disorder are not yet sufficient. This insufficiency could be associated with the overlap of symptoms between GAD and depressive disorders as well as with somatoform disorders, which are both quite frequently represented in German inpatient samples. Therefore, in this context the suitability of the PSWQ as a screening instrument for GAD would be a great convenience in order to not overlook this (comorbid) disorder in clinical daily routine. Nevertheless, as the specificity of pathological worry in GAD was questioned by several authors [17], [18], another goal was to examine the specificity of pathological worry for GAD by comparing the levels of pathological worry among psychotherapy inpatients with GAD, several other anxiety disorders and depression. In some studies, significant associations between PSWQ scores and rumination could be found [19]. Thus, an association of depressive symptoms with the process of worrying can be expected.


Method

Participants and diagnoses

This study refers to an unselected sample of N=293 psychotherapy inpatients who were treated in a German psychosomatic rehabilitation clinic (Paracelsus-Roswitha-Klinik, Bad Gandersheim, Germany). The study was approved by the Bundesversicherungsanstalt für Angestellte (BfA). Data were collected in the context of routine evaluation. Diagnostic classification was conducted by routine structured clinical interviews (Mini-DIPS) [20] according to DSM-IV [5] and ICD-10 [6] by trained psychotherapists under regular supervision. The primary diagnosis was defined as the diagnosis associated with the most severe impairment. Patients with substance abuse, psychotic or bipolar disorders, eating disorders, obsessive-compulsive disorders, and severe personality disorders (Cluster A and Cluster B) were excluded. Finally, N=237 patients were included in the present investigation. The mean age of the sample was 48.40 years (SD=9.06), 144 (60.8%) patients were female, and 140 (59.07%) were married. Regarding the primary diagnosis, N=18 (7.6%) patients of the sample fulfilled the criteria for GAD. N=32 (13.5%) patients had a primary or secondary GAD diagnosis. The frequency of all primary diagnoses is displayed in Table 1 [Tab. 1].

Assessment

The Penn State Worry Questionnaire (PSWQ) [11] is a self-report measure to assess pathological worry in GAD patients. Satisfactory reliability and validity could be demonstrated for the German version of the PSWQ [21], [22]. By adding up the value (five-point scale, range 1–5) of all 16 items (e.g., “I’m always worrying about something“) a score from 16 to 80 can be reached.

The Beck Depression Inventory (BDI) [23], [24] is a 21-item self-report measure which assesses the severity of depressive symptoms on a 4-point scale. The widely used instrument shows satisfactory reliability with an internal consistency of .88.

Procedure and statistical methods

In order to test whether patients with primary or comorbid GAD score significantly higher in the PSWQ than do Non-GAD patients, a one-tailed t-test was conducted and an effect size according to Cohen [25] was calculated.

The utility of the PSWQ as a screening instrument for GAD can be explored by a receiver operating characteristic (ROC) analysis [26]. The analysis produces a ROC curve in which the sensitivity is plotted against the specificity for each value of the test. The probability that a test will correctly classify the individuals is estimated by the area under the curve (AUC). The AUC is indexed from 0 to 1, values greater than 0.50 are interpreted as a probability greater than chance. Cut-off scores for optimal sensitivity (the score that optimized sensitivity without reducing specificity to less than chance), optimal specificity (the score that optimized specificity without reducing sensitivity to less than chance), and optimal sensitivity and specificity (the score that produced the best balance of sensitivity and specificity) were calculated for values of the PSWQ total score.

To investigate whether patients with GAD differ from patients with depressive disorders or other anxiety disorders in their amount of pathological worry measured by the PSWQ, a sub-sample of N=118 patients with anxiety disorders (F40/41) and depressive disorders (F32, F33, F34) was examined. Patients with other primary/comorbid disorders than the mentioned disorders were excluded. The sample of N=118 patients was divided into five subgroups (see Table 2 [Tab. 2]). Group 1 “Pure GAD“ (n=11) consists of patients suffering from GAD without any comorbidity. Group 2 “GAD & Depression“ (n=12) consists of patients with GAD and depressive disorders, but without any other anxiety disorder. Group 3 “Depression“ (n=50) consists of patients with a primary diagnosis of depression without a comorbid GAD or another anxiety disorder. The fourth group “Anxiety“(n=13) consists of patients with anxiety disorders without comorbid GAD or depression. Group 5 “Anxiety & Depression” (n=32) consists of patients with anxiety disorders (except for GAD) and comorbid depressive disorder.

Differences regarding pathological worry and depression measured by the PSWQ and the BDI across the five diagnostic subgroups were examined by conducting one-way ANOVAs. Bonferroni adjusted alpha (0.05/2) was α=0.025. The influence of depression on pathological worry was controlled by conducting an ANCOVA with BDI score as a covariate. All calculations were made using SPSS 12.0.


Results

The total sample of N=237 patients had a PSWQ score of 52.5 (10.41). By dividing the sample into patients with GAD as primary or comorbid diagnosis versus patients with all other diagnoses, GAD patients (n=32) reached a PSWQ score of 57.81 (8.42), while Non-GAD patients (n=205) had a PSWQ score of 51.67 (10.47). This difference is significant (one-tailed t-test, t(47)=3.701, p<.001) with a medium effect-size of d=.65 according to Cohen [25].

The ROC analysis (see Figure 1 [Fig. 1]) showed an area under the curve of AUC=.67 (p=0.02). According to Fresco et al. [13], three PSWQ cut-off scores were calculated. When sensitivity is optimized, a cut-off score of 51 with a sensitivity of .77 and a specificity of .51 resulted. If specificity should be more important, a cut-off score of 56 with a sensitivity of .52 and a specificity of .68 resulted. By simultaneously optimizing both sensitivity and specificity a cut-off score of 54 with a sensitivity of .58 and a specificity of .60 was found. To ensure the identification of positive cases, we considered the sensitivity of a screening instrument most important. When sensitivity is optimized, a cut-off score of 51 resulted and 77.4% of the GAD patients were correctly classified (25 patients), whereas 22.6% GAD patients were not identified (7 patients). In the group of Non-GAD patients only 51% of the patients (104 patients) were correctly classified, whereas 49% (101 patients) were falsely classified as GAD patients. Taken together, only 54.4% of the patients were correctly classified.

To compare GAD patients with patients suffering from depressive disorders and/or other anxiety disorders regarding pathological worry, measured by the PSWQ, a sub-sample of N=118 patients was examined. The total sub-sample reached a PSWQ score of 54.62 (10.14) and a BDI score of 22.88 (9.85). Correlation between PSWQ and BDI was r=.51 (p<.01). Table 2 [Tab. 2] shows the mean scores and standard deviations in the PSWQ and BDI in the diagnostic subgroups.

One-way ANOVAs showed significant differences between the subgroups in the PSWQ (F(4)=5.46, p<.001; Bonferroni post-hoc comparisons with p<.05: 2>4, 3>4, 4>5) as well as in the BDI (F(4)=17.18, p<.0001; Bonferroni post-hoc comparisons with p<.05: 1<2, 1<3, 1<5, 2>4, 4<3, 4<5). After conducting an ANCOVA to control for the influence of depression (BDI), no significant differences between the diagnostic subgroups could be found in the PSWQ (F(4)=1.53, p=.20).


Discussion

In this study, GAD patients reported a significant higher level of pathological worry with a medium effect measured by the PSWQ than do all other patients of the inpatient sample. Nevertheless, compared to earlier results, the GAD patients in this sample scored lower than those of other clinical samples [12], [13], although they scored higher than the normal population [21].

The ROC analysis demonstrated a weak result with only 54.4% of the patients correctly classified. This result suggests rather insufficient screening abilities of the PSWQ for GAD in this unselected sample of psychotherapy inpatients. If it is presumed that a specificity of pathological worries for GAD exists [9], than it seems as if the PSWQ is not able to indicate this specificity. So the instrument could be able to detect worry valid, but not specific enough to discriminate between different disorders.

Another possibility is that pathological worrying is not that specific for GAD [17], [18]. If pathological worrying can also be found in depressive disorders as well as in other anxiety disorders, the fact of nearly the half of all patients (49% of the sample) falsely classified as GAD patients in this study becomes comprehensible. This consideration is supported by the analyses of the five diagnostic subgroups. The subgroups exhibited significant differences regarding their level of pathological worry and depression. After controlling for the influence of depression, differences referring to pathological worry no longer existed. This result shows that pathological worry is not as specific for GAD as it is commonly postulated. Instead of that, our findings stress the meaning of comorbid depression in GAD and further show a relevant impact of depressive symptoms on the process of worrying. Thus, it could be useful to recognize GAD as a “matrix of anxious-somatic-depressive symptoms” [27] rather than to overemphasize the specificity of pathological worry in GAD.

Overall, it is important to note that the PSWQ was developed as a measure for severity of pathological worry and it was never intended to be used as a screening instrument for GAD. Because of the fact that GAD is a wider concept than pathological worry, an inability of the PSWQ to screen effectively for GAD should not automatically be regarded as a flaw of this instrument.

Although in general such a poor screening result could also be caused by weaknesses regarding the reliability and validity of the given diagnoses, the use of structured clinical interviews conducted by experienced psychotherapists under regular supervision speak rather against such an explanation for the presented results.

One shortcoming of our study is the small number of GAD patients in our clinical sample, which also suggests that GAD is not very frequent in inpatient settings. In addition, compared to other GAD samples, the GAD patients of our sample scored rather moderate on the PSWQ. Therefore, they could also be rather less prototypical for GAD. Another important limitation is the restricted generalizability, which is associated with the rather unique feature of the German health care system, where patients suffering from Axis I disorders can receive in-patient psychotherapy treatment. Thus, a simple generalisation to other inpatient settings outside of Germany would not be adequate.

Taken together, our results suggest that the PSWQ is rather inapplicable as a screening instrument for GAD in the context of German inpatient settings. Instead of that, in such a context the PSWQ could be used to discriminate high versus low worriers independent of the respective classificatory diagnoses. Therefore, the instrument could provide the building of e.g., symptom-orientated therapy groups as they are established in behavioural-medicine inpatient settings. Such an approach also makes allowance for the high rates of comorbidities in Axis I disorders.


Notes

Conflicts of interest

None declared.


References

1.
Kessler RC, Chiu WT, Demler O, Merikangas KR, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):617-27. DOI: 10.1001/archpsyc.62.6.617 External link
2.
Brown TA, Campbell LA, Lehman CL, Grisham JR, Mancill RB. Current and lifetime comorbidity of the DSM-IV anxiety and mood disorders in a large clinical sample. J Abnorm Psychol. 2001;110(4):585-99. DOI: 10.1037/0021-843X.110.4.585 External link
3.
Bruce SE, Machan JT, Dyck I, Keller MB. Infrequency of "pure" GAD: impact of psychiatric comorbidity on clinical course. Depress Anxiety. 2001;14(4):219-25. DOI: 10.1002/da.1070 External link
4.
Wittchen HU, Hoyer J. Generalized anxiety disorder: nature and course. J Clin Psychiatry. 2001;62 Suppl 11:15-9.
5.
American Psychiatric Association. Diagnostic and statistical manual of mental disorders DSM-IV. 4th ed., text revision. Washington, DC: American Psychiatric Association; 2000.
6.
Dilling H, Mombour W, Schmidt MH, editors. Internationale Klassifikation Psychischer Störungen: ICD-10 Kapitel V (F): diagnostische Kriterien für Forschung und Praxis. 4th ed. Bern: Huber; 2000.
7.
Marten PA, Brown TA, Barlow DH, Borkovec TD, Shear MK, Lydiard RB. Evaluation of the ratings comprising the associated symptom criterion of DSM-III-R generalized anxiety disorder. J Nerv Ment Dis. 1993;181(11):676-82. DOI: 10.1097/00005053-199311000-00005 External link
8.
Ladouceur R, Blais F, Freeston MH, Dugas MJ. Problem solving and problem orientation in generalized anxiety disorder. J Anxiety Disord. 1998;12(2):139-52. DOI: 10.1016/S0887-6185(98)00002-4 External link
9.
Hoyer J, Becker ES, Roth WT. Characteristics of worry in GAD patients, social phobics, and controls. Depress Anxiety. 2001;13(2):89-96. DOI: 10.1002/da.1021 External link
10.
Chelminski I, Zimmerman M. Pathological worry in depressed and anxious patients. J Anxiety Disord. 2003;17(5):533-46. DOI: 10.1016/S0887-6185(02)00246-3 External link
11.
Meyer TJ, Miller ML, Metzger RL, Borkovec TD. Development and validation of the Penn State Worry Questionnaire. Behav Res Ther. 1990;28(6):487-95. DOI: 10.1016/0005-7967(90)90135-6 External link
12.
Behar E, Alcaine O, Zuellig AR, Borkovec TD. Screening for generalized anxiety disorder using the Penn State Worry Questionnaire: a receiver operating characteristic analysis. J Behav Ther Exp Psychiatry. 2003;34(1):25-43. DOI: 10.1016/S0005-7916(03)00004-1 External link
13.
Fresco DM, Mennin DS, Heimberg RG, Turk CL. Using the Penn State Worry Questionnaire to identify individuals with generalized anxiety disorder: a receiver operating characteristic analysis. J Behav Ther Exp Psychiatry. 2003;34(3-4):283-91. DOI: 10.1016/j.jbtep.2003.09.001 External link
14.
Turk CL, Heimberg RG, Mennin DS. Assessment. In: Heimberg RG, Turk CL, Mennin DS, editors. Generalized anxiety disorder: advances in research and practice. New York: Guilford Press; 2004. p.219-247.
15.
Brown TA, Antony MM, Barlow DH. Psychometric properties of the Penn State Worry Questionnaire in a clinical disorders sample. Behav Res Ther. 1992;30(1):33-7. DOI: 10.1016/0005-7967(92)90093-V External link
16.
Ballenger JC, Davidson JR, Lecrubier Y, Nutt DJ, Borkovec TD, Rickels K, Stein DJ, Wittchen HU. Consensus statement on general anxiety disorder from the International Consensus Group on Depression and Anxiety. J Clin Psychiatry. 2001;62 Suppl 11:S53-8.
17.
Starcevic V, Berle D. Cognitive specificity of anxiety disorders: a review of selected key constructs. Depress Anxiety. 2006;23(2):51-61. DOI: 10.1002/da.20145 External link
18.
Starcevic V, Berle D, Milicevic D, Hannan A, Lamplugh C, Eslick GD. Pathological worry, anxiety disorders and the impact of co-occurrence with depressive and other anxiety disorders. J Anxiety Disord. 2007; 21(8):1016-27. DOI: 10.1016/j.janxdis.2006.10.015 External link
19.
Fresco DM, Frankel AN, Mennin DS, Turk CL, Heimberg RG. Distinct and overlapping features of rumination and worry: the relationship of cognitive production to negative affective states. Cognit Ther Res. 2002;26(2):179-188. DOI: 10.1023/A:1014517718949 External link
20.
Margraf J. Diagnostisches Kurzinterview bei psychischen Störungen (Mini-DIPS). Berlin: Springer; 1994.
21.
Stöber J. Besorgnis: Ein Vergleich dreier Inventare zur Erfassung allgemeiner Sorgen [Worry: A comparison of three questionnaires for the measurement of general worries]. Z Differ Diagn Psychol. 1995;16:50-63.
22.
Stöber J. Reliability and validity of two widely-used worry questionnaires: self report and peer convergence. Pers Individ Diff. 1998;24(6):887-90. DOI: 10.1016/S0191-8869(97)00232-8 External link
23.
Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J. An inventory for measuring depression. Arch Gen Psychiatry. 1961;4:561-71.
24.
Hautzinger M, Bailer M, Woral H, Keller F. Beck-Depressionsinventar [Beck Depression Inventory]. 2nd ed. Bern: Huber; 1994.
25.
Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: Erlbaum; 1988.
26.
Metz CE. Basic principles of ROC analysis. Semin Nucl Med. 1978;8(4):283-98. DOI: 10.1016/S0001-2998(78)80014-2 External link
27.
Stein DJ. Comorbidity in generalized anxiety disorder: Impact and implications. J Clin Psychiatry. 2001;62 Suppl 11:29-34.