Dr. Rizwan Malik, Clinical Lead for Radiology at the Royal Bolton Hospital in UK planned to test qXR, an artificial intelligence (AI) driven chest x-ray system developed by Qure.ai, a company based in Mumbai, India, when he noticed some of his patients needed to wait long hours for a specialist to examine their x-rays. However, before Dr. Malik could embark testing whether AI is efficient and safe in shortening patients’ waiting time, COVID-19 hit the country.
Early studies revealed some severe COVID-19 cases would exhibit unique lung anomalies that are similar to viral pneumonia. Since medical facilities are not able to accommodate everyone with COVID-19 symptoms for a RT-PCR test, looking at their chest x-rays become a triage shortcut. Dr. Malik decided to jump at the opportunity and proposed to re-tool qXR into a COVID-19 detection machine whereby AI will assist in the initial reading of the chest x-rays.
A tool that was born in an ongoing pandemic
Under normal circumstance, the re-tooling process may take months as hospitals need to inspect its design in great details, but not this time. Dr. Malik was given an immediate approval to use it at the Royal Bolton Hospital because of the unprecedented situation. “The aim of our deployment was for progression monitoring, for disease quantification and to aid clinical decisions process to signpost abnormalities to clinicians” Dr. Malik wrote to AIMed.
Dr. Malik said the qXR software was trained on millions of chest x-rays with abnormalities present in both tuberculosis and COVID-19 patients and had gotten CE approval. Besides, Qure.ai is experienced in developing inference algorithms for respiratory diseases and this gave them an edge in getting the AI validated on an open-source COVID-19 datasets that were made available. Dr. Malik believes all these gave him and his colleagues confidence to use the tool.
“Early detection is the key. The World Health Organization (WHO) has been pushing the mandate to test more as well. In an unprecedented situation such as we are in, the use of a tool like qXR greatly benefits our trainees. It boost sensitivity during triage amd minimizes errors.” Dr. Malik adds. In fact, Dr. Malik pointed out the diagnosis of COVID-19 is not made on chest x-ray findings or RT-PCR test alone. It’s based on a wide range of inputs including symptoms, one’s contact history, clinical suspicion and so on.
An AI algorithm stating whether one is COVID-19 positive or negative will not confirm the diagnoses or directly influence patient management. Likewise, a chest x-ray can be normal in a COVID-19 positive patient or a patient can be feeling unwell with an abnormal chest x-ray but not COVID-19 related. “In medicine, decisions are based on risks and benefits. There is no haste. Quick timely decisions are taken when we see benefits far outweigh the risks. There will always be risks even when an AI product is 100% ready for patients”.
Dealing with criticisms
Despite the knowledge, qXR was criticized recently for being error-prone, making COVID-19 patients looking worse than they were because the software was designed for tuberculosis in the first place. Dr. Malik felt the criticism was valid because it sides with patient safety rather than AI, which is an appropriate thing to do. He also highlighted the main concern revolved around the AI’s ability to quantify the percentage of lung infected by COVID-19.
Initially, Qure.ai received feedback that this percentage seemed to be low as compared to the estimates made by human radiologists. The algorithm had since been adjusted and the feature in question was fixed within two weeks. Dr. Malik regarded the incident as a positive development to reflect Qure.ai is not only open to collaborating on early validations but many of their work had also been peer-reviewed independently by different groups across the globe.
“In my conversation with them, I recall their algorithms being validated in almost 16 countries globally. They’ve trained their algorithms on datasets as large as 2.5 million chest x-rays. And in a lot of unique findings, the training of the algorithms was done in consultation and supervision with global experts in radiology”.
Dr. Malik said criticisms and challenges will not withdraw him from recommending other medical professionals to also look into readily available AI-driven tool and reconstruct them into something that can be used during the pandemic. “There is no reason to not use technology like AI in a situation like this when it’s been proved useful. Hybrid ‘human+AI’ workflows are here to stay. With our caseloads burgeoning, AI assist tools, remote patient management systems will help on combating the pandemic”.
Moving beyond the pandemic
Dr. Malik believed this pandemic proved to be a catalyst for a timely deployment in the face of a global health crisis. He and the Qure.ai are planning to introduce more features such as lines and tubes placement evaluation on chest x-rays, abnormal x-rays and more. Dr. Malik also plans to continue his original testing that was put on halt as a result of the pandemic.
“We need to remember a huge number of AI solutions predate the pandemic. Those which were re-tooled can of course be reverted to their original use. For solutions developed during the pandemic itself, the knowledge used to create them will not be lost and can be adapted with less time pressure for broader use cases.”
“We continue to learn important lessons on this journey. We are learning how to design internal evaluation. We are devising tools to get feedback from clinicians. A lot of guidelines on AI use and deployment are only starting to emerge as the pandemic evolved. So we take on board clinicians and junior doctors from A&E, ICU and wards with us.”
At the end of the day, “The COVID detection machine is driven by human intelligence, not AI. AI is a part of this machinery along with biochemistry, radiology etc.” Dr. Malik says.