Article Text
PDF
Abstract
Background: Problem based learning (PBL) is used increasingly in undergraduate medical education, but there are few postgraduate medical studies.
Aim: To compare SHO learning outcomes for a PBL course with a traditional didactic course.
Methods: As part of their protected teaching programme, 14 senior house officers (SHOs) were taught about paediatric dermatology using a traditional didactic course. Six months later, the new SHOs received a PBL course including small group teaching and a study guide. Both the traditional and the PBL group were assessed using multiple choice questions (MCQs), an objective structured clinical examination (OSCE), and pre- and post-course self-assessment sheets. SHOs completed course evaluation sheets.
Results: There was no significant difference in learning outcome between the traditional and PBL courses as assessed by the MCQs, OSCE, and self-assessment sheets. The PBL course was well appreciated by SHOs who liked variety in the teaching programme.
Conclusions: The PBL and traditional course had equivalent learning outcomes. PBL adds variety to junior doctor protected teaching programmes and can be a useful tool for doctors working shift patterns.
- LREC, local research ethics committee
- PBL, problem based learning
- MCQ, multiple choice question
- OSCE, objective structured clinical examination
- SHO, senior house officer
- junior doctor teaching
- medical
- paediatric dermatology
- postgraduate
- problem based learning
Statistics from Altmetric.com
- LREC, local research ethics committee
- PBL, problem based learning
- MCQ, multiple choice question
- OSCE, objective structured clinical examination
- SHO, senior house officer
Problem based learning (PBL) is considered to be one of the most important developments in health professionals education in the latter part of the twentieth century.1 PBL has been used in many universities, both in the UK and globally,2,3 and to a much lesser extent in postgraduate medical education.4,5 Senior house officers (SHOs) are required to have protected teaching each week. Since 1996, the paediatric teaching programme at Birmingham Heartlands Hospital has been subject to change due to introduction of partial and full-shift rotas. At least four of the 12 fulltime SHOs miss the protected teaching as two are on annual leave and two SHOs are on night shifts. A more self-directed programme would therefore be of benefit.
Traditionally, the teaching programme was consultant led. The first hour was a lecture/tutorial about a specific topic, and the second hour was ward based—seeing interesting patients, preferably ones with an illness covered by the talk. This teaching is somewhat “luck of the draw” as there may be no inpatients with the relevant condition. The didactic style of the teaching programme is not in keeping with recent trends in medical education which aim to promote learning which is interactive and at deeper cognitive levels on Bloom’s taxonomy.6 It would be preferable for SHOs to focus on areas where there is a gap in their knowledge, rather than sitting through an hour’s teaching, if much is known of what is said. As RCPCH tutor, I had evaluated the protected teaching programme. SHOs disliked always having the same format of teaching and had requested variety and diversity in the programme.
It is important, however, not to introduce change in the teaching programme just because PBL is a current trend in medical education. To change the teaching programme would involve considerable work including training of consultant teachers and overcoming natural reluctance to change. Most current SHOs are used to didactic forms of teaching and their response to a different style of teaching and learning needed to be fully evaluated. The purpose of this study was to compare SHO learning outcomes for a PBL course with a traditional didactic course.
METHODS
Subjects
These were the 14 paediatric SHOs at Birmingham Heartlands hospital (12 full-time and two flexible trainees). Courses did not run at the same time. The traditional course was held first followed by the PBL course six months later. Most SHOs on one year rotations changed hospital between the courses. While an SHO could attend the problem based learning course if he/she had already taken part in the traditional course, the SHO did not have a second assessment. Only data from SHOs who had attended one course was included in the statistical comparison of the two courses.
Course development
The course was planned using Harden’s 10 questions7 (box 1⇓). Prior to the course, an informal survey was carried out asking consultants (n = 6), SpRs (n = 6), and SHOs (n = 6) which paediatric dermatology topics SHOs would most want to learn about. In addition, the last 20 cases of skin disorder seen on the Paediatric Assessment Unit (PAU) were studied. A Delphi technique8 was proposed to select from the topics raised from these sources, but this was unnecessary as suggestions from the doctors surveyed and the PAU cases were all itchy red rashes. Hence, this became the title of the traditionally taught course and the focus of the PBL problem.
Harden’s 10 questions for curriculum planning
What are the needs in relation to the product of the training programme?
What are the aims and objectives?
What content should be included?
How should the content be organised?
What educational strategies should be adopted?
What teaching methods should be used?
How should assessment be carried out?
How should details of the curriculum be communicated?
What educational environment or climate should be fostered?
How should the process be managed?
Teaching methods for PBL course
The traditional and problem based courses are shown in box 2⇓. Small groups consisted of no more than four SHOs, each with a facilitator. The role of the facilitator was to help the discussion and to enable the SHOs to think around the problem for themselves rather than acting as a source of knowledge. All resources for the PBL course were catalogued in the postgraduate library which has 24 hour access. A 44 page (A4) study guide was included in the PBL course with notes on each condition, resource lists including contact details of the dermatology nurses, and self-assessment exercises.
Outline of the traditional and PBL courses
Traditional course
Pre-course self-assessment sheet
Interactive lecture on common causes of an itchy red rash
Bedside teaching on inpatient ward on children with skin disorders
Six weeks later assessments (OSCE, 25MCQs, post-course self-assessment) and completion of course evaluation sheets
PBL course
Pre-course self-assessment sheet
Two small facilitated groups. Problem of a 2 year old child with an itchy red rash for three weeks presented. SHOs define the problem and identify learning goals
Six weeks later, groups meet again to review whether the learning objectives have been achieved, share knowledge, and explore further learning needs
Six weeks later assessments (OSCE, 25MCQs, post-course self-assessment) and completion of course evaluation sheets
The assessments
The 25 question, one best answer MCQ paper included both higher order and fact recall questions. Three stations (atopic eczema, scabies, and psoriasis) were chosen for the OSCE as a balance between the need to have multiple problems and time constraints. OSCE stations lasted for 10 minutes, 6 minutes for the OSCE, 3 minutes for immediate feedback, and 1 minute to proceed to the next station. Paediatric secretaries acted as surrogate parents and had a list of salient features of the case. Standardised score sheets were filled in by the two tutors and another paediatric consultant (AR). Each OSCE station had a maximum score of 30. Due to the need to cover service, a different SpR acted as tutor in the PBL course to the traditional one. SHOs were given a brief information sheet about the patient, including a photograph.
SHOs filled in a self-assessment sheet before and after both courses, rating their knowledge on a five point Likert scale which ranged from poor to excellent. A course evaluation sheet was used to obtain the SHOs’ views about the teaching method and content of both courses, using a four point scale ranging from not satisfied to very satisfied. In addition, an evaluation sheet was completed at the end of the PBL course looking at usefulness of the study guide and suggested resources and SHO’s opinion of PBL. Local research ethics committee (LREC) approval was not required as at the time the study was initiated, the LREC (East Birmingham) was working to good practice guidelines as laid down by the Royal College of Physicians. These guidelines detailed the groups of people for whom ethics permission must be sought before undertaking research, and at the time of the study NHS employees were not listed (personal communication Dr Rose, Chairman of East Birmingham LREC).
Statistics
The SPSS statistical package was used to test whether there were any statistically significant differences between the traditional and the PBL course subjects in the mean OSCE scores, the mean MCQ scores, and the pre- and post-course self-assessments. Independent sample t tests were used. Two tailed p values were used and 95% confidence intervals (CI) quoted.
RESULTS
OSCE and multiple choice questions
Seven SHOs completed the traditional course, and a different seven SHOs completed the PBL course (table 1⇓). The experience of the SHOs was approximately equivalent on both courses, and they were equally split between general and neonatal paediatrics. There was no significant difference in the mean OSCE or MCQ results between the traditional and the PBL course (p = 0.8) (table 2⇓). The only statistical difference was between the mean scores of the atopic eczema OSCE station (p < 0.001). This statistical difference needs to be interpreted with caution due to marker and surrogate parent variation.
Comparison of the demographic details of SHOs on the traditional and PBL courses
Comparison of the mean OSCE and MCQ scores between the traditional and PBL courses
SHOs found the MCQ paper difficult after both courses. Scores ranged from 2 (8%) to 14 (56%). There was no time to give feedback on the MCQ questions in the PBL course due to emergency admissions. SHOs were offered the opportunity to see HG to discuss the MCQs in the next week, but none of them took up this offer.
Self-assessment sheets
Although a consistent improvement in scores was seen for atopic eczema, there was however no significant difference between the mean differences in self-assessment between the traditional and PBL courses (table 3⇓).
Comparison of scores between traditional and PBL pre and post course self-assessment sheets
SHO course evaluation sheet
There was general satisfaction with content in both courses. Five of the six SHOs on the PBL course who completed an evaluation sheet were very satisfied compared to three of seven SHOs on the traditional course. Suggested improvements for the courses are shown in box 3⇓.
Improvements suggested by the SHOs following the traditional and PBL courses
Traditional course
More dermatology sessions
Attend dermatology outpatient clinics
More clinical cases
Discuss more slides
More information on standard treatment regimes of common dermatological problems
PBL course
Ask the MCQs while showing slides
More practical
Teach wet wrapping
Bring creams/emollients to the teaching sessions for the SHOs to use
Use of study guide and library resources
Only three of the seven SHOs on the PBL course had used the study guide: two SHOs had looked at the guide 1–3 times and one SHO had looked at it 4–6 times. None of the SHOs had used any of the library resources, contacted the dermatology nurses, or attended the dermatology outpatient clinic.
DISCUSSION
There was no significant difference between the PBL and the traditional course in terms of learning assessed by MCQs, an OSCE, and the SHO’s self-assessment of knowledge. The difference noted between mean scores for the atopic eczema OSCE stations is most likely due to marker and surrogate parent variability; the importance of considering confounding factors has been highlighted in previous studies.9 Many authors have noted no difference in outcome between PBL and traditional courses in terms of assessment criteria.10–,13
This study has a number of limitations, including the small number of senior house officers taking part in the study (only half (7/14) of the SHOs in post managed to complete each course) and thus the ability to detect differences between learning outcomes of the two courses. Confounding factors were also present as outlined in the first paragraph of the discussion. SHOs were not randomised to the PBL or traditional didactic course as it would have been difficult to run both courses at the same time due to manpower constraints. Manpower was double to run the PBL course compared to the traditional one, and if the learning opportunities had been taken up by the SHOs, would have been considerably more.
What is already known on this topic
PBL has been introduced widely in undergraduate teaching, but there are few studies as part of postgraduate medical education
What this study adds
The PBL and traditional courses were equally as effective in terms of learning measured by an OSCE, MCQs, and self-evaluation in the protected teaching programme
PBL was liked by junior doctors as it added to variety in the programme
There is a need to familiarise both junior doctors and consultant teachers with problem based learning before introducing a programme
PBL is a useful tool when planning teaching programmes, taking into account partial and full shifts, and reduction in junior doctors’ hours
There was in general greater satisfaction with the PBL course compared to the traditional course. Academic equivalence, but a richer learning experience has been noted previously,14 as PBL is reported as providing a more challenging, motivating and enjoyable approach to learning.3,15,16
Four of the five improvements suggested by the SHOs following the traditional course were included in the PBL course. SHOs requested discussion of more slides. However, none of the SHOs on the PBL course looked at the tape slide collection in the library. Despite suggesting more practical dermatology, no SHO attended clinic or spent time with the dermatology nurses.
This study will be useful when considering the style of teaching in future programmes. PBL was successfully incorporated into a protected teaching programme with equivalent learning outcomes to traditional sessions. PBL was also well liked by the SHOs. Large group teaching methods are rapidly becoming outdated as the numbers of SHOs on day time shifts and junior doctors’ hours continue to decrease. It is highly likely that change will be needed in teaching programmes. Time does however need to be spent to familiarise a department with problem based learning prior to a PBL course.
Acknowledgments
I am grateful to the paediatric SHOs for participating in the courses and to staff at Birmingham Heartlands Hospital for their help; in particular, Drs Tina Newton and Sophie Wilne, SpRs, for acting as co-tutors; Dr Andrew Riordan, consultant paediatrician, for marking an OSCE station on both courses; Miss Megan Wood, Postgraduate Centre Administrator, for helping organise the courses; Miss Lesley Allen, Chief Librarian, for cataloguing the PBL resources; and the paediatric secretaries for acting as surrogate parents in the OSCE stations. I am also grateful to Dr Ian Campbell for statistical advice, and Dr David Wall, Deputy Postgraduate Dean, West Midlands for advice on the manuscript.
REFERENCES
Footnotes
Competing interests: none declared