gms | German Medical Science

EbM in Qualitätsmanagement und operativer Medizin
8. Jahrestagung des Deutschen Netzwerks Evidenzbasierte Medizin e. V.

Deutsches Netzwerk Evidenzbasierte Medizin e. V.

22.03. - 24.03.2007 in Berlin

Adjustment lowers Agreement and Precision of Results of Medical Examination

Meeting Abstract

Suche in Medline nach

EbM in Qualitätsmanagement und operativer Medizin. 8. Jahrestagung des Deutschen Netzwerks Evidenzbasierte Medizin e. V.. Berlin, 22.-24.03.2007. Düsseldorf, Köln: German Medical Science; 2007. Doc07ebm058

Die elektronische Version dieses Artikels ist vollständig und ist verfügbar unter: http://www.egms.de/de/meetings/ebm2007/07ebm058.shtml

Veröffentlicht: 15. März 2007

© 2007 Friedrich-Wedig et al.
Dieser Artikel ist ein Open Access-Artikel und steht unter den Creative Commons Lizenzbedingungen (http://creativecommons.org/licenses/by-nc-nd/3.0/deed.de). Er darf vervielf&aauml;ltigt, verbreitet und &oauml;ffentlich zug&aauml;nglich gemacht werden, vorausgesetzt dass Autor und Quelle genannt werden.


Gliederung

Text

Hintergrund

Rankings are used as result of assessment. An example for rankings of classes are marks. Marks result from the share of solution of stadardized tests. Adjustment ist usual as calibration, if a threshold error has to be corrected. The adjustment of factors as number of “staff of the faculty” and “logarithm of number of inhibitants of the city” lowers the agreement of rankings. Adjustment is a method for focussing, for demonstration not for increase of accuracy.

Methoden

Data of Zimmermann 2006 [1]

Calculation of Bland 1986 [2]

Precision of Calculation

Up to 10 07 iterations in the procedure of calculation of the t-value.

Plots of results

Figure 1 [Fig. 1], Figure 2 [Fig. 2], Figure 3 [Fig. 3].

Ergebnisse

Zimmermann’s comparison of results of the 4th semester examination of medical students (2006) are here investigated with the method of Bland (1986). Orinal ranking and adjusted ranking differ between –17 and +11 (n= 36). The space of two standard deviations count between –10 and 39. So the agreement of the ranking lists is 18 – 10 = 8 up to 18 + 39 = 49. This in not enough for equal use of original ranking and adjusted ranking.

The 0.95 confidence interval is 18,5 +/- (2.03 * 10.2) = [-2, +39]. And this interval is wider than the 36point original ranking. Because of the right rotating tilting of linear regression the weaker universities profit from the adjustment and the success of the best universities seems to bei smaller.

The connection between the success in objective tests and disturbance of factors which are part of the adjusted ranking is in matter of scientific and technological questions low: y = 0,78 x + 0,40 with R2 = 0,61.

Schlussfolgerung/Implikation

You must not use adjustment as a method for comparison of results of medical examination. Few intruments are needed for detection of pitfalls of rankings. Blands method is easy, quick and smart.


References

1.
Zimmermann R, Wegscheider K, van den Bussche H. Medizinische Fakultäten. Der Ausbildungserfolg im Vergleich (I). Dtsch Ärztebl. 2006;103(25):A-1732-A-1738.
2.
Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;307-10.