Rating criteria for assessing early child development measurement tool accuracy and feasibility for use in routine programmes
| Grading criteria | Definition | Rating | Meaning |
| A. Does the tool work? Psychometric properties and cultural adaptability of tool | |||
| 1. Validity | The degree to which a measure accurately assesses behaviours or abilities that reflect the underlying concept being tested. (16) | 3 | Validity ideally against educational outcomes up to age 5 with a standardised test, eg, Wechsler, equal to or above widely accepted threshold (eg, >0.7), statistically significant. |
| 2 | Validity somewhat below widely accepted threshold (eg, 0.5–0.7) against another performance-based tool, eg, Bayley III. | ||
| 1 | Some description/mention of validity but methods unclear or poor quality, below accepted threshold (eg, <0.5). | ||
| 0 | Inadequate result of validity, no statistical significance. | ||
| 2. Reliability | How consistently a measure produces similar results for a child or group of children with repeated measurements over a short period of time. (16) | 3 | Equal to or above widely accepted threshold (eg, >0.7) for measure tested at tool level, rigorous methods of testing, statistically significant ideally with kappa. (supplementary web appendix 1). |
| 2 | Somewhat below widely accepted threshold (eg, 0.5–0.7), rigorous methods of testing but in one continent only. | ||
| 1 | Some description/mention but methods unclear or poor quality or below accepted threshold (eg, <0.5). | ||
| 0 | Inadequate discussion of reliability, no statistical. | ||
| 3. Cultural adaptability | Modification of items, materials and procedures to fit the local context, such as translating items and changing words or pictures to reflect cultural differences. (16) | 3 | Easy modification of items, materials and procedures. |
| 2 | Minimum to moderate modification of items, materials and procedures. | ||
| 1 | Moderate to complex modification of items, materials and procedures. | ||
| 0 | Highly difficult modification of items, materials and procedures. | ||
| B. Can the tool be delivered? Practicality of administration | |||
| 1. Accessibility | Access to tool, including digital availability and costs to purchase and use the tool with equipment as required. Note: cost is allocated per child for 100 tests. Note: digital defined here as open access tool available online and app available. Note: cost does not include training costs, some tools may be freely available but require payment for a trainer to train the project team. | 3 | Tool, administration, scoring and interpretation, adaptation and training resources all available open access online with no intellectual property restrictions; no cost for tool, no additional equipment; app available. |
| 2 | Tool, administration, scoring and interpretation, adaptation and training resources all available open access online with no intellectual property restrictions, minimal cost to tool and/or equipment (≤US$10 per child), no app available. | ||
| 1 | Tool, administration, scoring and interpretation, adaptation and training resources all available online, but some intellectual property or other restrictions (eg, requirement for direct involvement tool authors/owners in research), moderate cost to tool and/or equipment (range >US$10 to ≤US$20 per child), no app available. | ||
| 0 | Not readily available online with intellectual property restrictions, high cost tool and equipment (range >US$20 per child), no app available. | ||
| 2. Training | Refers to duration of training, skill level of trainer and trainee and certification requirement. Note: duration of training does not include general field work. | 3 | Brief (≤1 hour), minimal (ie, non-specialist worker can train non-specialist worker), no certification requirement. |
| 2 | Moderate (>1 hour to ≤1 day), moderate (ie, non-specialist trainer) but requires more standardisation and training or direct assessments of children’s abilities that require moderate training and practice, no certification requirement. | ||
| 1 | Long (≤2 days), moderate (ie, non-specialist trainer) but requires more standardisation and training or direct assessments of children’s abilities that require moderate training and practice, may include certification requirement. | ||
| 0 | Long (≥3 days), specialist trainer and/or trainee, certification required. | ||
| 3. Administration time | Estimated time taken to administer the tool in completion, including scoring time. Note: when range is given an estimated median time for administration will be used. | 3 | ≤15 min, easy scoring. |
| 2 | >15 to ≤30 min, minimum to moderate scoring. | ||
| 1 | >30 to ≤60 min, moderate to complex scoring. | ||
| 0 | >60 min. | ||
| 4. Geographical uptake | Geographical use of the tool. | 3 | Used in at least three continents. |
| 2 | Used in two continents only. | ||
| 1 | Used in one continent only. | ||
| 0 | Used in one country only. | ||
| C. Individual-level tool only | |||
| 1. Clinical relevance and utility | Usability of tool for frontline worker for interpretation and response. | 3 | Easy interpretation, clear threshold for action and structure for counselling response and contextually appropriate referral. |
| 2 | Minimum to moderate interpretation, thresholds for action but unstructured response guidance and/or suggested response unlikely to be feasible in context. | ||
| 1 | Moderate to complex interpretation, no structured thresholds for action and/or suggested response unfeasible in context. | ||
| 0 | Highly technical interpretation (eg, with separate manual), no clear threshold for action, specialist referral response. | ||