Background In children with acute dyspnoea, the assessment of severity of dyspnoea and response to treatment is often performed by different professionals, implying that knowledge of the interobserver variation of this clinical assessment is important.

Objective To determine intraobserver and interobserver variation in clinical assessment of children with dyspnoea.

Methods From September 2009 to September 2010, we recorded a convenience sample of 27 acutely wheezing children (aged 3 months–7 years) in the emergency department of a general teaching hospital in the Netherlands, on video before and after treatment with inhaled bronchodilators. These video recordings were independently assessed by nine observers scoring wheeze, prolonged expiratory phase, retractions, nasal flaring and a general assessment of dyspnoea on a Likert scale (0–10). Assessment was repeated after 2 weeks to evaluate intraobserver variation.

Results We analysed 972 observations. Intraobserver reliability was the highest for supraclavicular retractions (κ 0.84) and moderate-to-substantial for other items (κ 0.49–0.65). Interobserver reliability was considerably worse, with κ<0.46 for all items. The smallest detectable change of the dyspnoea score (>3 points) was larger than the minimal important change (<1 point), meaning that in 69% of observations a clinically important change after treatment cannot be distinguished from measurement error.

Conclusions Intraobserver variation is modest, and interobserver variation is large for most clinical findings in children with dyspnoea. The measurement error induced by this variation is too large to distinguish potentially clinically relevant changes in dyspnoea after treatment in two-thirds of observations. The poor interobserver reliability of clinical dyspnoea assessment in children limits its usefulness in clinical practice and research, and highlights the need to use more objective measurements in these patients.