Diagnostic tests: as easy as I, II, III

Diagnostic testing keeps coming back to bite Archi, and that's not just because of my probability-based failure regarding a small relative and a missed diagnosis of congenital heart disease. No, the problem with diagnostic tests and their use and abuse remains difficult because the methods of research, the quality of research and the consequence of bad research aren't generally as high profile as therapeutic failures. Perhaps we could help a bit by adopting a similar approach to evaluating diagnostic tests as we do new drugs: go for phase I, II and III studies.

Phase I diagnostic studies look at the very lowest level, not toxicity in this case but ability to separate out the grossly affected from the clearly well, for example, interleukin 8 (IL-8) levels in those with meningococcal sepsis and admissions for routine circumcision. Phase II studies work towards defining what the right cut-off should be for a test in a group who are more reflective of the clinical setting we want it to be used in. Taking our example onwards, this would be the definition of a value of IL-8 in children with fever in an admissions unit which best defines those with sepsis. This is classically known as test ‘derivation’. Phase III studies then go further, and test the use of clear criteria in a group with uncertain diagnosis, compared with an effective reference standard, often described as test ‘validation’.

Failure to recognise the phase of test leads to real problems in clinical interpretation. For instance, a test that can tell the difference between high-functioning adults with autism and those in volunteering for a quick MRI of the head may well be useless in the community paediatrics clinic. Or may not – but there is no way of knowing with a phase I/II study. Showing that IL-6 >1000 pg/ml is almost invariably associated with gram-negative sepsis in one population of children with fever and neutropenia might mean that it's a great diagnostic test for all new admissions. But such a phase II study doesn't really prove this: it hints in the same way that radiological response of a tumour might mean that the drug is helpful, but this is not always true.

So, when you're next reading about an advance in diagnostic test technology, as well as the thoughts about prevalence, predictive values and likelihood ratios, try and peg the ‘phase’ first and it may save you lots of time and pain.

Bob Phillips, Centre for Reviews and Dissemination, University of York, York YO10 5DD, UK; bob.phillips@doctors.org.uk