Introduction Computer vision could be a valuable tool in medicine to track objects during a procedure. In an operating theatre computer vision could be used to track and monitor the use of surgical equipment, examples could be the scanning of trays pre and post operation to check instruments into and out of the operating theatre. The aim of this project was to create a system that utilises computer vision to identify surgical equipment. To discover the limitations of object detection systems, and the time required to train a model.

Method As part of a joint collaboration between GOSH and UCL computer science (CS) through the industry exchange network programme, CS students developed a platform for computer vision for reliable detection of surgical instruments using standard over-the-counter digital video and mobile phone cameras, in conjunction with an Object Detection model developed using Google’s TensorFlow Object Detection API. The desktop client application was built using PyQT. The client application sends individual frames to a cloud-hosted web service that handles the detection of objects and was built in Python providing a ‘real time’ object detection interface.

Results Using two similar surgical objects, curved and straight forceps, the team developed a system that accurately identified the instruments (>99% accuracy). The system could track items use and monitor the number of time an object as used and for how long. The training of models for object detection was however, lengthy and required a significant number of photographs to achieve a high accuracy of recognition.

Conclusion A model was developed and trained to recognise surgical instruments. The prospect of tracking objects in healthcare is exciting however the data required to train a model, and current camera capabilities, are limiting factors in the use of computer vision and further work is required.