Abstract Winner – Digital Medicine & Wearable Technology


Author: Regina Leung

Coauthor(s): Co-Authors: Andras Lasso PhD (1), Matthew Holden PhD Candidate (1), Gabor Fichtinger PhD, Director (1), Boris Zevin MD, PhD (2) (1) Laboratory for Percutaneous Surgery, School of Computing, Queen’s University, Kingston, Canada (2) Department of Surgery, Queen’s University, Kingston, Canada

Status: Completed Work

Funding Acknowledgment: Queen’s University, School of Medicine. The Great-West Life School of Medicine Studentship.

 Abstract Winner – Digital Medicine & Wearable Technology


Training towards proficiency in technical skills involving specific hand motions is prevalent across all disciplines of medicine and particularly relevant in learning surgical skills such as knot tying. The dominant skills training method has been apprenticeship where the trainee learns under the guidance of a mentor. However, decreasing resources in the healthcare system has prompted the need for integration of more self-directed learning.

We propose a new form of self-directed learning where a pair of holographic hands are projected in front of the trainee using the Microsoft HoloLens and guides them through learning various basic hand motions present in surgery and medicine. This study looks at the feasibility and effectiveness of using holographic hands as a skills training modality for learning hand motions compared to the traditional methods of apprenticeship and video-based learning.

A pair of holographic hands was created using surface mesh and animation software including MakeHuman, Blender, and Unity. Using the HoloLens, these holographic hands are projected in front of the trainee and used to guide them through learning various hand motions relevant to surgery and medicine creating a form of “self-directed apprenticeship” learning. To evaluate the feasibility and effectiveness of holographic hand learning, 9 participants were recruited and each learned 6 different hand motions from 3 different modalities (video, apprenticeship, HoloLens). Results of successful completion was evaluated by an external assessor. Feedback on effectiveness was also obtained through a questionnaire.

Participants had a considerable preference for learning from HoloLens and apprenticeship and a higher success rate compared to video-based learning. Across the 6 different hand motions learned, the highest success rate recorded for video-based learning participants was 33% compared to apprenticeship or HoloLens learning that scored at least a 67%. Furthermore, learning with holographic hands was shown to be comparable to apprenticeship in terms of both effectiveness and success rate. Overall, 78% of participants agreed that HoloLens learning was comparable to learning by apprenticeship. However, more participants still selected apprenticeship as a preferred learning method compared to HoloLens (67% apprenticeship vs 56% HoloLens). As augmented reality technology matures and becomes more mainstream, these differences are expected to decrease over time.

This initial pilot study has demonstrated the feasibility of learning hand motions from a pair of holographic hands using the HoloLens. Furthermore, the study has shown that the holographic learning modality has the potential to mimic the apprenticeship experience and is preferred over video-based learning. The results are encouraging to suggest a new effective form of self-directed apprenticeship learning that combines the advantages of existing learning modalities. This technology can be further developed and applied to learning a variety of technical skills in medicine and surgery. Work continues toward implementing this technology in knot tying and suture tutoring modules in our undergraduate medical curriculum.