Case Study 1: Evaluation strategies devised to assess a recently deployed artificial intelligence (AI) driven chest x-ray system.

Background:, a Mumbai based company created qXR when they noticed patients needed to wait long hours for a specialist to examine their x-rays. At the peak of the pandemic in the middle of April, qXR was retooled by Dr. Rizwan Malik, Clinical Lead for Radiology at the Royal Bolton Hospital in UK into a COVID-19 detection machine as early studies revealed some severe cases would exhibit unique lung anomalies that are similar to viral pneumonia. Since medical facilities are not able to accommodate everyone with COVID-19 symptoms for a RT-PCR test, looking at their chest x-rays become a triage shortcut.

What have been done: According to Dr. Shaista Meraj, Consultant Radiologist at Royal Bolton, a team making up of consultant radiologists, Picture archiving and communication system (PACS) managers, business managers as well as Dr. Malik himself was formed. They designed a structured evaluation method to assess the accuracy, sensitivity and specificity of qXR in actual clinical deployment and whether it can assist in patient management. The method includes getting clinicians’ feedback; an internal evaluation, patient engagement and education, and post-market surveillance with the AI company.

Dr. Meraj did not indicate how many clinicians were involved in the evaluation process but what the team found was majority of the surveyed clinicians agreed AI tool is easy to use. They expressed the instant analysis generated by qXR was useful. Half of them indicated they would follow their own interpretations and the other half said they would consult a colleague when discrepancy happens. Clinicians also touched on the barriers for adoption and the need for a shift in culture to regard AI as a colleague rather than an intruder.

There were two parts to the internal evaluation. One is on analytical validation, which determined the reliability and accuracy of qXR and the other is clinical acceptance, looking at the safety aspect of the tool. The team would like to find out the technical readiness to integrate qXR with the existing PACS; correlation between analyses made by qXR and COVID diagnoses; progression monitoring capability and so on. The team is now planning some sort of active engagement and patient education to continue explore the benefits and limitations of AI in different perspectives.

Case Study 2: Leading a healthcare center to be the forefront of innovation for AI implementation

What have been done: Dr. Amrita Kumar, Consultant Radiologist and Clinical Lead AI at Frimley Health NHS Foundation Trust said it all started with a pressing need to improve radiologists’ workflow and the belief that AI could be of help. So, they formed an AI working group to engage with the right expertise and stakeholders including senior leadership and patients and to set out a vision for the initiative. Dr. Kumar believes in the importance of aligning their AI implementation goals with the local and national AI strategies as she realized although the working group was excited about bringing changes, not everyone shares the same enthusiasm.

The patient-centeredness nature of this AI initiative also pushes the working group to collaborate with other social care and secondary care services to raise awareness. Frimley Health then partners with vendors and academic institutions to get access to algorithms; potential funding, and help to implement AI.

“For AI to take place, you need data, you need domain expertise and you need computational power. We have the data, which we are trying to get organize into a connected care platform, working alongside our integrated care system. We have the domain expertise because we have the clinicians who want to be involved. What we don’t have is the computational power and at this point, we thought it is sensible to look for partnerships,” Dr. Kumar explains.

For example, one of the partnerships looked into the biomedical and clinical needs for patients who have been tested positive for COVID-19 and matches their diagnoses with imaging data. Essentially, the focus is to promote health, diagnosis and system efficiency so they will continue to pick projects that are backed by strong evidence and elicit a real cultural shift. On the other hand, empower patients and make them understand and be happy about the progresses that are put in place.

Case Study 3: Achieving routine clinical deployment

Background: There are many market approved products claimed to be beneficial for patients and they can be found across the whole medical domain yet they are not being used routinely. Haris Shuaib, Senior Physicist in Magnetic Resonance at Guy’s and St. Thomas’ NHS Foundation Trust and Topol Digital Fellow and his team aimed to address this particular challenge.

What have been done: The team started with the technical considerations, specifically, system integration. They spoke with the industrial partners in their ecosystem and find out if the AI tools would work in the way clinicians are delivering care and they would like information and reports to be received and generated. The team continued with staff training; ensuring the tools are producing data and results that radiologists are comfortable dealing with and the process is done effectively and accurately.

The next step, which Shuaib regarded as the most important, is sustainability. When the AI tool moves away from the research project stage and being deployed in an actual clinical setting, the team needs to make sure it remains robust and produces the same value. To ascertain that, Shuaib and team ensure the tool meets both the general (i.e., not specific to any AI technology) and specific (i.e., AI technology targeting certain area like mammography or head CT) requirements.

This routine clinical deployment framework was used by Shuaib and team for the past few years until the NHSx, the new digital unit of NHS heard of the effort and an evaluation center was set up. “I believe strongly those of us who have the capabilities and resources, have a duty to invest and then share this type of work to the benefits of everyone else,” Shuaib says. As the team continues with pilot projects and settles on the tools to be deployed, they are also building an in-house competency so that not only the radiologists will be supported but also clinicians in other domains.

“The whole point is to start off a cycle, with feedback from clinicians to ensure good technologies are created. Good technologies will then lead to good data and better technologies. So, it’s not about doing AI properly but doing medicine properly,” Shuaib adds.

If you are keen on finding out more of such real-world AI deployment stories and strategies, do not miss the upcoming AIMed Radiology virtual conference on 5 November (Thursday). Register or obtain a copy of the agenda here today!


Author Bio

Hazel Tang A science writer with data background and an interest in the current affair, culture, and arts; a no-med from an (almost) all-med family. Follow on Twitter.