Objectives To identify the unanswered research questions in paediatric preventive care that are most important to parents and clinicians, and to explore how questions from parents and clinicians may differ.
Design Iterative mixed methods research priority setting process.
Setting Toronto, Ontario, Canada.
Participants Parents of children aged 0–5 years enrolled in a research network in Toronto, and clinicians practising in Toronto, Ontario, Canada.
Interventions Informed by the James Lind Alliance’s methodology, an online questionnaire collected unanswered research questions in paediatric preventive care from study participants. Similar submissions were combined and ranked. A consensus workshop attended by 28 parents and clinicians considered the most highly ranked submissions and used the nominal group technique to select the 10 most important unanswered research questions.
Results Forty-two clinicians and 115 parents submitted 255 and 791 research questions, respectively, which were combined into 79 indicative questions. Most submissions were about nutrition, illness prevention, parenting and behaviour management. Parents were more likely to ask questions about screen time (49 parents vs 8 clinicians, p<0.05) and environmental toxins (18 parents vs 0 clinicians, p<0.05). The top 10 unanswered questions identified at the workshop related to mental health, parental stress, physical activity, obesity, childhood development, behaviour management and screen time.
Conclusion The top 10 most important unanswered research questions in paediatric preventive care from the perspective of parents and clinicians were identified. These research priorities may be important in advancing preventive healthcare for children.
- Preventive medicine
- priority setting
- evidence-based medicine
- research methods
- patient perspective
Statistics from Altmetric.com
What is already known on this topic?
When selecting important research questions for study, researchers and healthcare consumers often have different priorities.
There are few instances where patients, caregivers and clinicians have set paediatric research priorities together.
What this study adds?
This study identifies the top 10 most important unanswered research questions in paediatric preventive care from the perspective of parents and clinicians.
Studying these questions will help to fill evidence gaps with relevant data in this understudied population.
Many preventive care recommendations for children lack high-quality supportive evidence.1–3 Of 54 recommendations by the US Preventive Services Task Force (USPSTF) for child or adolescent preventive care, 20 (37%) were based on poor-quality evidence.4 The Bright Futures Guidelines published by the American Academy of Pediatrics has also acknowledged numerous gaps in the evidence for paediatric preventive care.4 5
Given the extent of these knowledge gaps, how should preventive care research topics for children be prioritised? It has been well documented that the priorities of researchers may not align well with the priorities of patients and providers.6–9 Patient involvement in setting research priorities may lead to the funding of research that is of higher relevance to patients.10 11 Although patients and providers are increasingly recognised as important research partners, a systematic map of priority setting studies found that only 19% of 258 studies involved both patients and clinicians in the process.12
The James Lind Alliance (JLA) was developed to prioritise health research on topics that matter to patients, caregivers and clinicians.13 The JLA developed a priority setting partnership (PSP) process that brings these three groups together to identify the most important unanswered research questions.14 The primary objective of this study was to identify the 10 most important unanswered research questions in paediatric preventive care from the perspective of parents and clinicians using the JLA approach. The secondary objective was to identify similarities and differences between research questions submitted by parents and clinicians.
Steering Group and online questionnaire development
Five paediatricians and five parents were recruited for the study’s Steering Group from within the TARGet Kids! (The Applied Research Group for Kids) primary care research network, a university-affiliated network whose aim is to advance the scientific basis for chronic disease prevention and develop innovative solutions for common children’s health problems (see online supplementary appendices 6 and 7).15 An online questionnaire was developed to obtain research priorities in paediatric preventive care from parents and clinicians. To encourage all possible submissions, we first used an open-ended question that asked participants to identify questions about keeping children healthy as they grow up. Participants were then asked to submit questions about specific preventive care topics (see online supplementary appendix 2). These topics were identified after a search of Medline, Embase and The Cumulative Index to Nursing and Allied Health Literature (CINAHL) for broad preventive care topics identified in the literature as important to parents and clinicians. Ten relevant studies suggested eight categories of questions of concern to parents16–25: physical activity, nutrition, growth, parenting, behaviour, mental health, development and preventing childhood illness. The content validity of the questionnaire was reviewed and consensus was reached by the Steering Group. The study protocol was approved by the Research Ethics Boards of the Hospital for Sick Children and St Michael’s Hospital.
Collecting and identifying unanswered research questions
The questionnaire was distributed online using the REDCap platform hosted by the Hospital for Sick Children Research Institute.26 Questions were collected from parents of children aged 0–5 years through the TARGet Kids! research network, as well as from paediatricians and family physicians through TARGet Kids! and the Ontario Medical Association (OMA), the organisation representing physicians in the province of Ontario, Canada, between February and November 2014. Eight hundred parents whose e-mail addresses were provided to TARGet Kids! were asked to complete the study questionnaire online. Eighty clinicians either within the TARGet Kids! network or working in Toronto, Ontario, were contacted; in addition, the questionnaire link was distributed via e-mail to members of the OMA Pediatrics Section. Demographic information was collected from participants, and clinicians were asked about their specialty and whether they work in primary care. Responses were submitted anonymously and no personally identifiable health information was collected.
Submitted questions were reviewed by three members of the research team (ML, CSB and JLM). Submissions that were not research questions, were unrelated to prevention or were considered to already be answered by high-quality evidence (such as a systematic review or randomised controlled trial) were removed. The remaining questions were grouped by similarity and were used to generate indicative questions that represented the theme of the original submissions. For example, the question ‘How much screen time is appropriate for children?’ was generated by combining questions such as ‘How much time should a child spend in front of the TV?’, ‘How much TV/screen time is considered too much?’ and ‘What is the maximum daily amount of time a 3 year old should be in front of a computer/TV/iPad?’ Parent and clinician questions were kept separate at this stage.
A review of relevant literature in Medline, Embase and CINAHL was performed to assess the evidence related to each indicative question. Indicative questions were considered unanswered if there was no systematic review on the topic, if there was a systematic review which indicated insufficient evidence or if there was a grade I (insufficient) level of evidence in one or more preventive health guidelines (including Bright Futures,5 the USPSTF recommendations,4 the Rourke Baby Record,27 the Greig Health Record,28 and position statements from the Canadian Paediatric Society and American Academy of Pediatrics). Questions were reviewed by three team members (ML, CSB and JLM) and unanswered indicative questions were retained. A Master List of unanswered indicative questions was generated by combining similar unanswered questions submitted by parents and clinicians (see online supplementary appendix 3). Master List questions were ranked by the number of respondents whose questionnaire submissions fell within the scope of each indicative question (the ‘parent and clinician score’). For example, if a question about a certain topic was submitted by 15 parents and 10 clinicians, the indicative question was given a parent and clinician score of 25. The top-ranked question was the one that had the highest number of submissions from parents and clinicians (the maximum score).
Steering Group members individually ranked the 79 questions in the Master List from most to least important on a numerical scale. The questions were ranked based on the Steering Group members’ perception of their importance to paediatric preventive care research. This step was taken because there were more parent respondents to the initial questionnaire than clinician respondents, and parent submissions therefore made up a larger part of the Master List score than clinician submissions for some questions. A ‘Steering Group score’ was generated from the sum of these rankings and used to reorder the Master List questions by importance. Thirty-nine questions of the original 79 questions on the Master List were retained as the Interim List. These 39 questions were felt to be most important to the parent and clinician questionnaire respondents and to the Steering Group. In addition to the top 30 questions ranked by the Steering Group, 9 additional questions were retained to preserve the most frequently submitted questions by parents and clinicians (see online supplementary appendix 4). Questions 34, 35, 36 and 39 from the Master List were not retained, while questions 41, 42, 50 and 58 were included in the Interim List (see online supplementary appendix 5).
Priority setting workshop
An in-person priority setting workshop was held in December 2014 using a modified nominal group technique to achieve consensus between parents and clinicians on the top 10 most important unanswered research questions.29 Parents and clinicians were recruited via e-mail through the TARGet Kids! research network and the OMA. One week before the workshop, the Interim List was distributed to attendees, who individually ranked the unanswered research questions in order of importance prior to attending the workshop.
Parents and clinicians were evenly divided into four groups, each with a facilitator. During two morning sessions, the four groups ranked the 39 questions on the Interim List in order of importance by consensus. Each group was instructed to identify pairs of questions they felt were thematically similar, if indicated. The question perceived to be the most important of the two questions in similar pairs was ranked, and the question considered less important of the two was not assigned a rank by that group. The mean ranking for each question was calculated across the four small groups, and the top 20 questions were retained for the afternoon sessions. During the first afternoon session, participants were assigned to four different small groups, and each group discussed and ranked the top 20 questions. The mean ranking for each question was again calculated across the four small groups, and a final session which involved all participants was used to reach consensus on the top 10 most important unanswered research questions.
After the workshop, attendees were asked to rate on a scale of 1 (poor) to 5 (excellent) their overall experience and how effectively the modified nominal group technique allowed the group to achieve consensus on the top 10 unanswered research questions.
Descriptive statistics were calculated for parents and clinicians who responded to the online questionnaire. Questions in the Master List were organised into 11 thematic categories by 3 team members (ML, CSB and JLM). To identify similarities and differences in the response between categories of unanswered research questions in the Master List, we used the χ2 test of independence or Fisher’s exact test. Associations were evaluated between the type of respondent (parent or clinician) and the submission of at least one unanswered research question for that category (yes or no). The false discovery rate method was used to account for multiple testing by adjusting the raw p values obtained from the χ2 tests.30 Data analysis for this study was performed using SAS V.9.3 software of the SAS System for Windows.
Respondents to the online questionnaire included 115 parents and 42 clinicians (see table 1). Parents and clinicians submitted a total of 791 and 255 questions about paediatric preventive care research, respectively. Most submissions pertained to nutrition and obesity (18% of parent questions, 20% of clinician questions), illness prevention (17% and 19%), and parenting and behaviour (15% and 16%). Parents were more likely than clinicians to submit questions about screen time and media exposure (49 parents vs 8 clinicians, p <0.05), as well as environmental toxins (18 parents vs 0 clinician, p<0.05) (see table 2).
Of the questions submitted, 397 parent questions (50%) and 151 clinician questions (59%) were unanswered. As some questions were similar in both groups, 57 unique questions from parents and 45 from clinicians were combined into the 79 indicative questions of the Master List (see figure 1). The top-ranked indicative question based on the total number of submissions was ‘What are effective strategies for parents to discipline children?’ which had been submitted by 10 clinicians and 40 parents. Other commonly submitted unanswered research questions included the impact of electronic devices on child development (32 parents and 5 clinicians), and effective interventions to promote healthy sleep (19 parents and 8 clinicians) (see table 3).
Priority setting workshop
Ten parents (8 mothers and 2 fathers) and 18 clinicians (12 paediatricians, 5 family physicians and 1 nurse) participated in the priority setting workshop. Over the course of the three small-group sessions, some similar questions from the Interim List were paired by each group separately. For example, during the first morning session, one group felt that the question ‘What are effective strategies for early identification of language difficulties?’ was similar to but less inclusive than the question ‘What are effective methods for screening for developmental delay in children?’ Because the first question was perceived to be less important than the second, it was not ranked by this group, which meant its final rank was lower than the second question. As well, two wording changes were made during the workshop that affected the final top 10 list. First, the word ‘discipline’ in question 21 from the Interim List was changed to ‘behaviour management’ so that the final version of the question read ‘What are effective strategies for behaviour management in children?’ As well, question 6 from the Interim List was rephrased to read ‘What nutritional factors affect child behaviour?’
At the conclusion of the final session, participants reached a consensus on the top 10 unanswered questions (see table 2). The top question was ‘What are effective strategies for screening and prevention of mental health problems?’ The other top five questions were about interventions to increase physical activity, the impact of daycare attendance on health and development, interventions for obesity prevention in young children and methods to promote social skill development.
In the post-workshop questionnaire (57% response frequency), respondents ranked their overall experience 4.3 out of 5. When asked how well they felt the nominal group technique allowed the priority setting goal to be reached, the average score was 3.7 out of 5. Six respondents provided comments. One respondent noted that parents and clinicians may understand the research questions differently and suggested providing a brief paragraph on what is known about each question. Another respondent remarked that as a parent, they deferred to clinicians during parts of the group session due to a perceived lack of knowledge about certain questions. Two other respondents felt the process efficiently accomplished the project objectives, and that parents were a crucial part of the workshop.
This study identified the top 10 unanswered research questions in paediatric preventive care from the perspective of parents and clinicians using a process informed by the JLA PSP.14 Parents were more likely to identify questions about screen time, media exposure and environmental toxins than clinicians. We found little information about environmental toxins in existing preventive care guidelines, with the exception of Bright Futures recommendations about lead in paint.5 The questions in the top 10 list included topics such as mental health, child development, obesity prevention and physical activity. The USPSTF identified these topics among seven ‘high priority evidence gaps’ related to children and adolescents in a report to the US Congress in 2014.31 Two categories in the USPSTF report were not identified by the respondents in our study, namely cancer prevention and child maltreatment. This study identified topics that were not addressed by the USPSTF report, such as environmental toxins and poverty.
Strengths of our study include the number of submissions, our focus on preventive care and the use of a published methodology to elicit priorities from both parents and clinicians. While most previous PSPs have focused on a specific condition such as asthma, this study addressed questions about the broader field of paediatric preventive care. The priority setting exercise also attempted to balance viewpoints from stakeholders. This was accomplished through multiple sessions with different groups, and by having moderators with considerable focus group and priority setting experience ensure that all participants’ viewpoints were heard. There was a good questionnaire response rate for parents; other PSPs have had similar response frequencies such as PSPs on vitiligo and tinnitus (18% and 19%, respectively).32 33
Limitations of the study include that parent respondents were primarily well-educated mothers living in higher income households. Their priorities may be different from fathers or parents with lower socioeconomic or educational status, and our results may not be generalisable to these and other groups. A similar study should be performed with different sample populations to assess generalisability. As well, although this study focused on the 0–5 years old age group, future research may seek to give a voice to older children and teenagers who can reflect on keeping children healthy as they grow up. Finally, some parents felt that the extent of their knowledge may not have allowed them to make fully informed decisions on the relative importance of unanswered questions during the workshop. Future PSPs might study whether providing information on the existing evidence base for each question would allow parents/patients to participate more confidently in the priority setting workshop.
To our knowledge, no other study has brought parents and clinicians together to identify the most important priorities for preventive care research for children. This is particularly important in paediatric research because children are an understudied and vulnerable population.34–36 We hope that the research priorities identified through this study will help researchers, funders and professional organisations advance preventive healthcare for children.
The authors would like to acknowledge the study participants and the members of the Steering Group for their hard work through the multiple steps of this priority setting process, as well as the parents, clinicians and facilitators who attended the final ranking workshop. The authors would like to thank Yang Chen and Dr Gerald Lebovic, biostatisticians at the Applied Health Research Centre at St Michael’s Hospital, for their assistance. We also acknowledge the Pediatricians Alliance of Ontario for their support of the priority setting process and final ranking workshop.
Contributors ML conceptualised and designed the study and data collection instruments, coordinated data collection, carried out the initial analyses, coordinated the final workshop, drafted the initial manuscript, reviewed and revised the manuscript, and approved the final manuscript, as submitted. CSB
, JLM and AL reviewed and revised the study protocol and data collection instruments, supervised data collection, participated in the final study workshop, reviewed and revised the manuscript, and approved the final manuscript, as submitted. SS
reviewed the study design, supervised data collection, participated in the final study workshop, reviewed and revised the manuscript, and approved the final manuscript, as submitted.
Competing interests None declared.
Ethics approval Hospital for Sick Children Research Ethics Board, St. Michael’s Hospital Research Ethics Board.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.