In the medical field “competence” refers to a physician’s ability to plan and execute safe patient management, and requires integration of knowledge, skills, and attitudes involved in clinical tasks (for example, eliciting and organising a patient history, performing a physical examination, ordering and interpreting laboratory data, giving treatment and follow up, and communicating with and educating patients). Since the 1970s and 1980s much attention has been given to the need for development and assessment of professional competencies in many medical disciplines.¹

The only way to truly determine competence is to observe physicians caring for patients in a variety of clinical settings and under different circumstances.² However, the limited time available for patient encounters and potential medical-legal problems make such assessment of health personnel difficult in practice. In addition, such performance based examinations are commonly subjective and inconsistent, with considerable variation among examiners. It was only after the introduction of standardised patients (SPs) 40 years ago,³ that performance based assessment with objective scoring became possible and reliable. SPs have been used in several settings, for example, objective structured clinical examinations (OSCE), and standardised patients examination, as supported by the Accreditation Council of Graduate Medical Education and American Board of Medical Specialties.⁴ The Medical Council of Canada has used SPs in their national licensing examination since 1992, and the National Board of Medical Examiners of the United States will soon be incorporating them into the Medical Licensing Step 2 Examination.

The use of child SPs to assess the clinical competence of paediatricians is difficult because of the variable physical and psychometric properties inherent in this age group, and the possibility of psychosocial problems. Children have been used as SPs for more than 20 years,⁵ but only a very limited number of studies with a small number of child SPs have been reported in the paediatric literature. It is therefore difficult to find models for developing successful child SP programmes.

This paper reviews the current status of the use of child SPs in the assessment of paediatric clinical competence. We highlight the problems and offer potential solutions for resolving these problems, so as to improve performance based examinations.

DEFINITIONS

When used for assessment, a simulated patient is a well person trained to simulate an illness in a standardised way. An actual patient is someone with a disease who is trained to present his or her own illness in a standardised way. SPs include both simulated and actual patients. Because the information they give is standardised, examiners can observe and judge test takers’ performance relatively objectively. Child SPs are defined as SPs younger than 18 years old. Standardised mothers or families are not included in this review, although they are commonly involved in paediatric clinical assessments. OSCE is a multiple station examination which measures a variety of elements of clinical competence. The test taker rotates from station to station where he or she is tested on a specific element, such as taking a focused history, performing part of a physical examination, interpreting lab results, etc. However, SP based clinical skill assessment (CSA) deals with the whole patient, involving a number of different skills, and which requires the test taker to evaluate a number of possible problems that a physician would likely see in practice.

METHODS

English articles that describe the use of child SPs for assessing clinical competence in paediatrics were collected. The Medline database was searched using the following key words combined with paediatrics and then combined with children: simulated patient, standardised patient, programmed patient, OSCE, clinical competence assessment, and clinical skill assessment. All the retrieved articles were reviewed for their relevance. This search produced 19 relevant papers.^{5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23}

RESULTS

Use of child SPs in paediatric assessment

In paediatrics, most reports regarding the use of children as SPs involved OSCE. Only two SP based CSA experiences were described.^6,16 Based on the four reports that described the contents of the stations in their paediatric OSCE,^7,9,10,14 child SPs were used in 12–27% (3–9 stations) of all stations used in an OSCE (see table 1). The patient ages ranged from infancy to adolescence. In most cases, there were duplicate child SPs for any particular station so that the children had a chance to rest. Regarding the content of the simulation, only older children had to learn a scenario. They were specifically coached to simulate patients with abdominal pain, hearing loss, secondary amenorrhea, and history of swollen ankles and dark urine. Young children with consistent, stable physical findings and cooperative dispositions were asked to perform the simulations for the respiratory, cardiovascular, and abdominal examinations. The examination of an infant with fetal alcohol syndrome required mainly inspection. Healthy infants were used for newborn physical examinations.

Compared with OSCE, CSA is a longer session to assess comprehensive clinical skills, and thus fewer reports of CSA appear in the paediatric literature. Lane and colleagues⁶ reported their experience in using child SPs for CSA. Eleven children between 7 and 16 years of age were recruited for 7 out of 10 stations. They portrayed patients with enuresis, asthma, child abuse, dark urine, limpness, headache, and fatigue. Hergenroeder and colleagues¹⁶ reported on use of a child to assess the residents’ physical examination techniques for the ankle and knee.

Another approach that avoids the use of children has been to present clinical problems for a second opinion or to use only adults to present a simulated history. For example, examinees have been asked to take a history from the mother (or other family member) without the presence of the child.^6,14 Lane and colleagues⁶ found that the stations without children were perceived to be as realistic as other stations.

Qualitative evaluation of the examinations using child SPs

The quality of an examination is often indicated by its validity, reliability, and feasibility. Validity is the extend to which an instrument generates meaningful scores. Validity can be defined by face validity (for example, the appearance of the test instrument regarding its validity), content validity (for example, the adequacy with which the test samples the domain of measurement), and construct validity (for example, the inferences correctly made from the operational definition of a variable to its theoretical constructs). Reliability is the estimated consistency that scores can be reproduced.

Reliability

Mastell et al used child SPs in some OSCE stations and showed an inter-station reliability α of 0.34, 0.12, 0.54, and 0.69.²⁴ Lane et al reported reliability in the range of 0.64 to 0.81.⁶ The wide α range was attributed to the heterogeneity of tasks at different stations. When the number of stations increased, α was calculated to be greater than 0.8.^7,14

Validity

Scores in the paediatric OSCE correlated significantly with scores on the multiple choice question examinations, written problem solving,²⁴ and the residents’ in-training evaluation reports (ITERs),^8,24 showing good validity. The validity of the content was variable, depending on the number of stations used. However, most participants perceived that child SPs enhanced realism and face validity.^6,7 The paediatric OSCE can therefore test various aspects of paediatrics that are essential educational objectives. Most studies reporting inclusion of child SPs in clinical examinations have shown construct validity.^6,8

Feedback

Patient feedback given after the OSCE or SP based clinical examinations indicated that the learners thought these methods were more effective than any other teaching methods.⁷ Even young children were able to give an overall patient satisfaction rating for the encounters.⁶

Overall researchers regard clinical examinations that include child SPs as valid and reliable assessments, which also provide effective feedback from the “patients”.

Cost

The expense and time consumed for hiring and training SPs were considerably more than for any other form of assessment. The cost was estimated to be around $70 per examinee in Canada and $50 per examinee in the United States.^8,25,26 The specific cost per child SP was not reported, but it is clear that using them will likely increase the cost of clinical assessments. In addition, children nearly always need to be accompanied by a parent, who may need to make childcare or school transport arrangements for siblings and may also need to take time off work.

Ethical issues and child SPs’ responses

Age of the child SPs is the major ethical concern. Children are often too young to understand what they are expected to portray as an SP. In addition, their behaviour may be inconsistent and difficult to control. Therefore, some physicians regard the use of child SPs as a form of child abuse.⁷ Ethical questions that have been raised include allowing newborns to be examined by several examinees, the morality of asking parents to consent to the use of their children as SPs when there may be no benefit to the child, who is eligible to give consent for the child, and whether the child understands what is expected of him when he is used. Therefore, Hilliard and Tallett in Toronto⁸ decided not use any children in their paediatric OSCE, while Lane and colleagues⁶ said that they would not use children younger than 7 years old.

From the viewpoint of children and their families, the results of the SP experience varied among different age groups. Younger children were at greater risk for negative effects, such as exhaustion, fear, and misunderstanding of the simulation.⁵ Older children reported that they acquired important skills, self assurance, and helpful information through the simulation. They reported learning some things about biology, how physicians are trained, and what physicians do.^5,6 Most of these children and their parents said they enjoyed the experience and the opportunity to earn some money.^5,6,11 One group found that simulation of emotional problems may be difficult for some adolescents, so their maturity needs to taken into consideration during selection.⁵ The University of Toronto Psychiatric OSCE Project has initiated research to examine the impact on adolescents portraying high and low emotional roles in an OSCE.¹⁸

Examinees’ responses towards the child SPs based clinical examinations

Residents strongly agreed that clinical examinations using child SPs measured important clinical objectives¹⁰ and that these assessments were challenging, realistic, and enjoyable.⁶ However, some examinees did raise concerns about fairness.⁶

DISCUSSION

Shortcomings in using children as SPs

Despite positive results in a number of reports on the use of child SPs, there are still a number of shortcomings that limit their use, as compared with adult SPs. Children often cannot portray an actual patient well enough to convince the examinees they are dealing with a realistic situation. It is also difficult to train children to play a particular scenario. Therefore, children can only simulate a limited variety of illnesses. Children cannot be examined repeatedly by a large number of examinees. Even the best behaved child is likely to become bored, tired, or uncooperative. This can affect their performance, which can change over time and between students, resulting in a less objective, standardised assessment. In addition, it is difficult to find ideal child SPs because children are more likely than adults to have acute illnesses with physical findings that change or resolve rapidly. Parents are often concerned about the child’s feelings during the simulation, and some are reluctant to allow their child to miss school for a day. Children cannot fill out long reports and may not be able to give adequate feedback about the examination. In general compared with use of adult SPs, use of child SPs is more expensive, time consuming, and difficult to carry out. Given these limitations, a comprehensive clinical assessment procedure should include more than one method of testing (for example, chart simulated recall oral examination, 360 degree clinical examination, global rating of a live or recorded performance). Suitable methods should be chosen according to the objectives of the examination, constraints on time and resources, available testing methods, and acceptability to all concerned.

Overcoming problems in using child SPs

It has been suggested that care should be taken in selecting children as SPs and that specific SP roles should be assigned with a view to benefiting the child as well as the examinee.⁵ In addition to consistent, reproducible physical findings, the children selected should have cooperative dispositions and be able to portray a patient successfully. This is necessary as the child is simulating not only a model of a particular disease but also of a typical patient encounter. The examinee’s interaction with the SP is included in the assessment. The type of roles assigned to child SPs should match their developmental age. The closer the simulated role is to the child’s age and personality, the less time is required for training. Young children can do well presenting actual own physical problems or complaints they have had, and, with adequate coaching, they may be able to portray patients with hyperactivity, attention deficit disorder, headaches, or stomach pains. Older children (>13 years old) can be coached to simulate complicated behaviours or emotional problem they themselves have not encountered.

The use of child SPs should be limited to subjects that cannot be satisfactorily measured using other methods, so that young children do not have to be repeatedly examined. After a brief simulated examination, the assessment may include evaluation of radiographs, laboratory results, ECGs, CT scans, pulmonary function tests, and review of an actual clinical course. For some purposes, there are good substitutes for a real patient, such as child mannequins, video recordings (showing various abnormal movements, cerebellar signs, examination of a floppy infant, and various types of seizure), audiovisual stations for examination of the cardiorespiratory system, slides or photographs of dermatological findings, and microscopic findings.¹³ Use of adults to simulate parents is a good way to assess history taking.

When child SPs are necessary for assessing clinical skills, the following guidelines are helpful:

• Limit the number of examinations to less than 10 per child per session.

• Avoid scheduling a child for both a morning and an afternoon session.

• Try to find children with similar findings who can substitute for one another, and switch them after every four or five examinees.

• Avoid using inpatients, who are already subject to multiple examinations, or the same outpatients repeatedly.

• Permission for a child to act as an SP should first be obtained from a parent or guardian.

• If there are concerns about the child’s response prior to the assessment, input from a school counsellor may be helpful before the events.

• Ask the parents or guardian to accompany a child less than 13 years old.

A database of SPs should be carefully kept, which may help in finding substitute or duplicate SPs. If an examination proves to be distressing or difficult for a child SP, their name should be removed from the active database. The child’s school schedule can be included in the database so as to avoid time conflicts. Ideally, serving as an SP should be scheduled to coincide with school breaks. Box 1 summarises tips for development of paediatric clinical assessments that use child SPs.

Conclusion

Paediatric clinical assessments using child SPs are generally regarded as valid and reliable, and often provide effective feedback. However, because of the practical difficulties and ethical issues involved in using child SPs, this assessment method should be limited to those areas that cannot be satisfactorily measured by other methods. Through meticulous attention to detail and careful planning, the use of child SPs can result in a clinical assessment that is objective for the examinee, non-traumatic for the child SPs, and interesting for both.