Article Text

Download PDFPDF
75 Computer vision for object detection; machine learning-based identification of surgical equipment
  1. Benedict Chan1,
  2. Shirin Harandi2,
  3. Daiana Bassi3,
  4. Sue Conner3,
  5. Yun Fu1,
  6. Dean Mohamedally1,
  7. Gemma Molyneux3,
  8. Graham Roberts1,
  9. Neil Sebire3
  1. 1UCL
  2. 2UCL Student
  3. 3GOSH


Introduction Computer vision could be a valuable tool in medicine to track objects during a procedure. In an operating theatre computer vision could be used to track and monitor the use of surgical equipment, examples could be the scanning of trays pre and post operation to check instruments into and out of the operating theatre. The aim of this project was to create a system that utilises computer vision to identify surgical equipment. To discover the limitations of object detection systems, and the time required to train a model.

Method As part of a joint collaboration between GOSH and UCL computer science (CS) through the industry exchange network programme, CS students developed a platform for computer vision for reliable detection of surgical instruments using standard over-the-counter digital video and mobile phone cameras, in conjunction with an Object Detection model developed using Google’s TensorFlow Object Detection API. The desktop client application was built using PyQT. The client application sends individual frames to a cloud-hosted web service that handles the detection of objects and was built in Python providing a ‘real time’ object detection interface.

Results Using two similar surgical objects, curved and straight forceps, the team developed a system that accurately identified the instruments (>99% accuracy). The system could track items use and monitor the number of time an object as used and for how long. The training of models for object detection was however, lengthy and required a significant number of photographs to achieve a high accuracy of recognition.

Conclusion A model was developed and trained to recognise surgical instruments. The prospect of tracking objects in healthcare is exciting however the data required to train a model, and current camera capabilities, are limiting factors in the use of computer vision and further work is required.

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.