It used to be a lot more complicated. When Laszlo Igali, consultant histopathologist at the Norfolk and Norwich University Hospital began his career, most pathologists had to rely on table lamps and mirrors to give them additional light source and support to clearly study the microscopic structure of tissues (i.e., histology) and structures and functions of living cells (cytology). Technology has, of course, changed it all, but pathology still has its own unique set of challenge: that is time and space.
Most pathologists examine tissues that are taken from a specific body part at the course of a malady. These tissues are often small in size, probably half the size of a human red blood cell. As they were removed from their natural environment and kept stagnant at the time of removal, these tissues will soon lose track on the prognosis of the malady. This makes it challenging for pathologists to search for solutions they wish to find via studying these tissues.
They have thus turned to artificial intelligence (AI) and augment intelligence (AuI) for an answer. Although these new technologies assist clinicians in their decision-making process by making sense of a large amount of data, they lack the abilities to decide on an appropriate research question and to determine on an area within a 3D object which deserves a second look.
Furthermore, while digital scanning and display technologies are slowly replacing microscope to facilitate the visualization of morphology (i.e., study of the forms and relationships of different living structures), some tissues still need to be processed with eosin (i.e., red fluorescent dye) and haematoxylin (i.e., purple biological stain). This slows down the whole diagnosis process.
The role of mixed reality
These are gaps which Igali thought virtual reality (VR) and augmented reality (AR) could fill. “Imagine if we could view tissues inside the patient, using in vivo microscopy to link cellular features to radiological images,” he wrote on The Pathologist. Igali added it used to be a dream for many AR companies to “spatially interact with floating holograms” but now it’s gradually becoming a reality. The projects which he had been involved in recently are mostly towards that direction.
For example, AR headset is now capable of memorizing users’ actions, taking real-time 3D photographs, transcribing video calls and creating useful information from different data sources, to decide on where to cut a tissue sample and collect its volumetric measurements All these will ascertain patient safety.
In another project, Igali and his team are working on the world’s first AI-controlled AR patient and its dialogue management system. Once it’s successful, it will bring medical teaching into a whole new level as trainees can holistically interact, speak, and even work at a microscopic level without involving an actual human patient.
Igali believes as VR headset becomes lighter or even negligible in the near future, medical professionals can be trained practically anywhere. They can follow through a holographic model using their phones and learn to perform procedures or taking measurements. On the other hand, professionals can also 3D record the skin lesion they had performed earlier and played back for reference later on.