Objective: To evaluate the usefulness of an objective structured clinical examination (OSCE) for assessing and providing feedback to postgraduate residents in pediatrics.
Design: A 5-station OSCE given in 1996, based on the educational objectives of a general pediatric training program. Each station assessed the residents' interviewing and history-taking skills with a standardized patient. The results were correlated with those of the in-training evaluation reports.
Setting: The Department of Paediatrics, University of Toronto Faculty of Medicine, Toronto, Ontario.
Participants: Forty-three of 61 pediatric residents: 14 first-year, 12 second-year, 8 third-year, and 9 fourth-year residents.
Main outcome measures: Scores for each of the 5 stations were broken down into 15 points for the check-list, 5 for the global assessment, and 10 for the written postencounter question, for a total score of 150. The in-training evaluation report ratings were converted to a 5-point numerical scale, ranging from 1 (unsatisfactory) to 5 (outstanding).
Results: The mean OSCE score for the 43 pediatric residents was 104.9. Although the residents in their senior year scored higher, there was no statistically significant difference among the 4 years for the total OSCE score or for any of the 5 stations. The fourth-year residents' scores on the postencounter questions were significantly (P < .05) higher than the first-year residents' scores. Two residents scored less than 60%. The internal consistency of the 5-station OSCE was limited (r = 0.69). Residents received verbal feedback at the conclusion of the OSCE, and they received their scores when they were calculated. The mean overall in-training evaluation report score for all 61 pediatric residents was 3.9. There was a moderate, but statistically significant, correlation between the overall mean OSCE results and the overall mean in-training evaluation report scores (r = 0.45).
Conclusion: The OSCE can provide a useful formative evaluation of postgraduate residents, but the usefulness of the evaluation data and the feedback must be balanced with the logistic difficulties and expense.