Responses

other Versions

PDF
‘The Score Matters’: wide variations in predictive performance of 18 paediatric track and trigger systems
Compose Response

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
Author Information
First or given name, e.g. 'Peter'.
Your last, or family, name, e.g. 'MacMoody'.
Your email address, e.g. higgs-boson@gmail.com
Your role and/or occupation, e.g. 'Orthopedic Surgeon'.
Your organization or institution (if applicable), e.g. 'Royal Free Hospital'.
Statement of Competing Interests

PLEASE NOTE:

  • Responses are moderated before posting and publication is at the absolute discretion of BMJ, however they are not peer-reviewed
  • Once published, you will not have the right to remove or edit your response. Removal or editing of responses is at BMJ's absolute discretion
  • If patients could recognise themselves, or anyone else could recognise a patient from your description, please obtain the patient's written consent to publication and send them to the editorial office before submitting your response [Patient consent forms]
  • By submitting this response you are agreeing to our full [Response terms and requirements]

Vertical Tabs

Other responses

Jump to comment:

  • Published on:
    Response from the authors
    • Susan M Chapman, Gulf Regional Clinical Manager Great Ormond Street Hospital for Children NHS Foundation Trust
    • Other Contributors:
      • Jo Wray, Health Psychologist
      • Kate Oulton, Research Fellow
      • Samiran Ray, Research Fellow
      • Mark J Peters, Professor of Paediatric Intensive Care

    We are pleased to see the interest shown in our article by Drs Cheung and Lachman, but cannot agree with their assertion that our research ‘misses the point’.
    Despite widespread use, there remains limited research on the effectiveness of paediatric early warning systems (PEWS) in detecting deterioration in hospitalised children. Our paper sought to establish if there were statistically significance differences in performance between 18 published systems. Trigger systems were out-performed by scoring-systems in this relevant but narrow assessment. Our conclusion emphasizes that it is unclear what factors account for these differences in performance.
    Dr Cheung and others feel this observation of statistical inferiority of trigger system is not merited and the observed differences may be influenced by the scoring threshold selected. Dr Cheung illustrates this by comparison to the threshold selection of the C-reactive protein test as an indicator of inflammation. We found this to be rather confusing. The outcomes of trigger-based systems are, by their very nature, dichotomous. Whilst there is always a trade-off between sensitivity and specificity for scoring-based systems, the same does not apply to trigger-based systems. The system is either triggered or not. We note that Dr Cheung does not offer data to support his preference for trigger systems.
    We agree that managing deterioration in children is complex. However it is hard to imagine how this would be im...

    Show More
    Conflict of Interest:
    None declared.
  • Published on:
    Content vs Context in scoring systems
    • Damian T Roland, Honorary Associate Professor in Paediatric Emergency Medicine On behalf of the PUMA Study group (https://pumastudy.com/)

    The need to improve outcomes in child health is not disputed, especially in the UK where performance lags behind the rest of Europe [1]. Mechanisms which detect, and respond, to the deteriorating child in an effective manner should be validated and shared so we welcome further research by Chapman et al. [2] which demonstrates the complexity of producing tools which do this. However, we support the concern raised by Cheung and Lachman [3] in ensuring that appropriate conclusions are drawn from this work. As a research group funded to investigate these systems, from both a quantitative and qualitative viewpoint, we would like to highlight some of the dangers in use of terminology in this area. The concept that systems may be a better paradigm than scores (i.e. the amalgamation of observations into binary or composite measures which determine pre-defined actions) is not new [4] and it is already recognised that both afferent and efferent limbs are vital in order to complete what Joffe described as the, “chain of events needed to improve response to inpatient deterioration.” We note Maconochie and Lillitos use the term PES (Paediatric Early Warning System) and differentiate trigger systems from PEWS (Paediatric early warning scores). It is not however clear why trigger systems are treated as separate from PEWS as the literature contains numerous examples of where a trigger type system has been labelled as a PEWS [5]. While we entirely agree there is a challenge in labelling...

    Show More
    Conflict of Interest:
    Our research group is delivering an NIHR Funded HS&DR programme to produce an evidence-based, system-wide approach to improving paediatric patient safety and reducing mortality in UK hospitals.
  • Published on:
    "Human factors matter" - Statistical analysis of performance of trigger systems misses the point
    • Ronny Cheung, Consultant Paediatrician Evelina London Children's Hospital, Guy's & St Thomas' Foundation NHS Trust
    • Other Contributors:
      • Peter Lachman, Consultant Paediatrician & CEO

    Chapman et al (1) present a valuable evaluation of the performance characteristics of 18 commonly used paediatric early warning systems. They observed that the performance of the 12 “scoring” systems (where cumulative component values for vital signs are used to identify thresholds for escalation of care) was superior to 6 “trigger” systems (where breaching set thresholds for one or more vital signs lead to escalation without the need for adding numerical scores), based on sensitivity, specificity and area under the operating curve (AUROC). Although they do not specifically claim that this finding should be extrapolated to suggest that all scoring systems outperform trigger systems, this is the implication both in the results and discussion section. Indeed, the associated editorial by Lillitos & Maconochie confirms this implied conclusion, when they state that “In conclusion…overall, PEWS perform better than Trigger systems.” (2)

    We contend that this is an erroneous and misleading conclusion and far outstrips the scope and methodology of the study. Firstly, the findings are related to the performance of 16 specific tools and no comment can be made about whether it is the Trigger or the Score aspects which are responsible for this difference. Using the analogy of a therapeutic trial, there can be no basis to conclude that this is a “class effect” rather than specific to each tool. Secondly, the authors themselves recognise that it is the thresholds for escalation...

    Show More
    Conflict of Interest:
    None declared.