Table 1

Rating criteria for assessing early child development measurement tool accuracy and feasibility for use in routine programmes

Grading criteriaDefinitionRatingMeaning
A. Does the tool work? Psychometric properties and cultural adaptability of tool
1. ValidityThe degree to which a measure accurately assesses behaviours or abilities that reflect the underlying concept being tested. (16)3Validity ideally against educational outcomes up to age 5 with a standardised test, eg, Wechsler, equal to or above widely accepted threshold (eg, >0.7), statistically significant.
2Validity somewhat below widely accepted threshold (eg, 0.5–0.7) against another performance-based tool, eg, Bayley III.
1Some description/mention of validity but methods unclear or poor quality, below accepted threshold (eg, <0.5).
0Inadequate result of validity, no statistical significance.
2. ReliabilityHow consistently a measure produces similar results for a child or group of children with repeated measurements over a short period of time. (16)3Equal to or above widely accepted threshold (eg, >0.7) for measure tested at tool level, rigorous methods of testing, statistically significant ideally with kappa. (supplementary web appendix 1).
2Somewhat below widely accepted threshold (eg, 0.5–0.7), rigorous methods of testing but in one continent only.
1Some description/mention but methods unclear or poor quality or below accepted threshold (eg, <0.5).
0Inadequate discussion of reliability, no statistical.
3. Cultural adaptabilityModification of items, materials and procedures to fit the local context, such as translating items and changing words or pictures to reflect cultural differences. (16)3Easy modification of items, materials and procedures.
2Minimum to moderate modification of items, materials and procedures.
1Moderate to complex modification of items, materials and procedures.
0Highly difficult modification of items, materials and procedures.
B. Can the tool be delivered? Practicality of administration
1. AccessibilityAccess to tool, including digital availability and costs to purchase and use the tool with equipment as required.
Note: cost is allocated per child for 100 tests.
Note: digital defined here as open access tool available online and app available.
Note: cost does not include training costs, some tools may be freely available but require payment for a trainer to train the project team.
3Tool, administration, scoring and interpretation, adaptation and training resources all available open access online with no intellectual property restrictions; no cost for tool, no additional equipment; app available.
2Tool, administration, scoring and interpretation, adaptation and training resources all available open access online with no intellectual property restrictions, minimal cost to tool and/or equipment (≤US$10 per child), no app available.
1Tool, administration, scoring and interpretation, adaptation and training resources all available online, but some intellectual property or other restrictions (eg, requirement for direct involvement tool authors/owners in research), moderate cost to tool and/or equipment (range >US$10 to ≤US$20 per child), no app available.
0Not readily available online with intellectual property restrictions, high cost tool and equipment (range >US$20 per child), no app available.
2. TrainingRefers to duration of training, skill level of trainer and trainee and certification requirement.
Note: duration of training does not include general field work.
3Brief (≤1 hour), minimal (ie, non-specialist worker can train non-specialist worker), no certification requirement.
2Moderate (>1 hour to ≤1 day), moderate (ie, non-specialist trainer) but requires more standardisation and training or direct assessments of children’s abilities that require moderate training and practice, no certification requirement.
1Long (≤2 days), moderate (ie, non-specialist trainer) but requires more standardisation and training or direct assessments of children’s abilities that require moderate training and practice, may include certification requirement.
0Long (≥3 days), specialist trainer and/or trainee, certification required.
3. Administration timeEstimated time taken to administer the tool in completion, including scoring time.
Note: when range is given an estimated median time for administration will be used.
3≤15 min, easy scoring.
2>15 to ≤30 min, minimum to moderate scoring.
1>30 to ≤60 min, moderate to complex scoring.
0>60 min.
4. Geographical uptakeGeographical use of the tool.3Used in at least three continents.
2Used in two continents only.
1Used in one continent only.
0Used in one country only.
C. Individual-level tool only
1. Clinical relevance and utilityUsability of tool for frontline worker for interpretation and response.3Easy interpretation, clear threshold for action and structure for counselling response and contextually appropriate referral.
2Minimum to moderate interpretation, thresholds for action but unstructured response guidance and/or suggested response unlikely to be feasible in context.
1Moderate to complex interpretation, no structured thresholds for action and/or suggested response unfeasible in context.
0Highly technical interpretation (eg, with separate manual), no clear threshold for action, specialist referral response.