Introduction Advancements in Augmented Reality technologies allow for the generation of three-dimensional (3D) models of anatomical structures from MRI or CT scans. We present the HoloRepository 2020 Proof of Concept as an open source FHIR-enabled research demonstrator using openly available imaging data.

Method We used the latest Machine Learning (ML) algorithms in the field for organ segmentation with a supporting structure for future ML algorithm revisions. The Cancer Imaging Archive was used for testing and integration, providing over 30 million radiology images for over 70 different anatomical structures. With this, a model training process that will allow hospitals and clinical groups to build out organ segmentation models on their own DICOM sets is anticipated.

Results This new version has 3 main editions: the Cloud HoloRepository 2020 (CH20), Intel NUC optimized HoloRepoistory 2020 (NH20), and HoloRepository 2020 Viewer (H20V). These facilitate new components such as an organ segmentation library, enhanced augmented reality experiences through cameras and multi-monitor displays, and a synthetic medical imaging data platform. CT and MRI DICOM scans of the brain, lungs, chest, abdomen and kidneys are rendered as a 3D view using a pre-trained model and the latest techniques for organ segmentation. A step-by-step guide on how to carry out the implementation procedure from research to deployment is laid out, enabling easy integration of newly available segmentation algorithms and methods.HoloRegistration enables simplified over-the-body tracking of the holographic views, achieving an Augmented Reality clinical education experience.

Conclusion The viewer and Intel NUC application allows for HoloRepository to be run on a local laptop or workstation, providing easier and faster access. Optimisations from this study have reduced generation times from 30 seconds to 3 seconds and cloud hosting costs by 66%. Future research into volumetric measurements and 2D cross-sectional tracking may aid diagnostics and assist navigation during surgery.