gms | German Medical Science

One popular procedure in the medical student selection process are multiple mini interviews (MMIs), which are designed to assess social skills (e.g., empathy, assertiveness) by means of brief interview and role-play stations. However, it remains unclear whether MMIs reliably measure desired social skills or rather general performance differences (across/within-stations). Here, we provide a detailed investigation into the construct validity of MMIs.

In Study 1 (3 samples, n=376 applicants, 144 raters) we used Bayesian generalizability analyses to show that the largest amount of reliable MMI variance was accounted for by station-specific and general performance differences between applicants. These findings suggest that MMI ratings are less social skill-specific than originally conceptualized.

In Study 2 we aimed at investigating the underlying structure and predictive power of specific interpersonal behaviors shown within and across MMI stations. For this, the whole MMI (n=203 applicants) was videotaped and trained experts rated over 50 specific behavioral cues (e.g., upright posture, humorous statements, aggressive gestures, smiling) across three stations. Structural equation models show that these behavioral differences can be reduced to the broader constructs of Agency, Communion, emotional stability, and intellectual competence, which all influence subsequent judgments (irrespective of desired social skill). There was no evidence of bias due to gender, attractiveness, age, or mere self-presentation.

Taken together, these two studies offer the following implications for the reliable measurement of social skills:

1.

Focus on only one social skill per station that is assessed at multiple stations.

2.

Focus on distinct social skills that allocate to either agency communion, emotional stability, or intellect.

3.

Specifically design stations that evoke maximum variability in the respective social skill.