Dr. Rubeta Matin, NHS Consultant Dermatologist, reveals the challenges of setting up a new national skin database to support the development of dermatological AI in the UK
It’s common knowledge that the chances of survival increase dramatically if melanoma is detected and treated early. However, many algorithm-based applications that claim to identify potentially dangerous-looking pigment on the skin have not been formally and appropriately validated in intervention studies.
There are also not many systematic and rigorous reviews to discover the true accuracy of these skin cancer diagnosing algorithms, especially those that were tested in an artificial research setting that may not be representative of the real world. It’s reasons like these that drive dermatologists to question whether the false assurance given by these applications may delay individuals from seeking medical advice.
Last February, a new study published in the BMJ revealed mobile applications that assess the risks of suspicious moles may not be reliable enough to detect all forms of skin cancer. Researchers warned that the present regulatory process for these applications “does not provide adequate protection to the public”. In the US, the FDA has not approved any such application, but SkinVision and SkinScan were regulated as Class 1 medical devices and deemed as low to moderate risk to users in Europe.
Dr. Rubeta Matin, Consultant Dermatologist and Honorary Senior Clinical Lecturer in Dermatology at the Oxford University Hospitals NHS Foundation Trust also raised similar concerns at the recent AIMed Clinician Series: Imaging event. There isn’t sufficient evidence to account for the effectiveness of AI interventions in dermatology. Moreover, many AI interventions focus on differentiating between benign and malignant skin lesions with a particular emphasis on melanoma diagnosis, rather than addressing a clinically unmet need or challenge.
“Unlike radiology, there are no DICOM standards for image-taking in dermatology,” Dr. Matin says. “Most of the time, images captured by patients using their mobile phones either turned out to be blurry or taken too far away and under the wrong lighting. These variables not only influenced dermatologists’ judgment but also the domain fails to accumulate the quality data required to train an efficient AI model.”
Dr. Matin believes having an AI-driven tool that improves the quality of images, is fulfilling a real clinical unmet need. “The topics that everyone is publishing are not necessarily the most important areas we need to tackle,” she says. Often it’s the opposite. Dermatological AI is not progressing at the same pace as other medical subspecialties because we don’t have well-curated datasets and we are unsure of the accuracy of the developed tools.
“There were quite a few publications that mentioned how performances of algorithms varied according to the skin type and skin tones of individuals from different ethnicities and across different settings like private practice and tertiary centers. Until we begin to routinely collect images from a diverse and representing population to reduce biases and assess AI-driven tools in settings that they will be deployed, we have a long way to go.”
Another concern raised by Dr. Matin revolved around the impact of AI adoption on non-life-threatening cancers. “Diagnosis has improved over the years, and we are now able to detect pre-malignant lesion,” she says. “But what happens when we introduce an even more efficient tool that will exponentially increase the number of diagnoses and cancer which is not life-threatening. How should we set the new gold standard?”
To set some pioneering standards for dermatological AI, last December the British Association of Dermatologists (BAD) set up a multidisciplinary AI Working Group to support “appropriately regulated and governed uses of AI interventions to enhance clinical practice”. The Working Group aimed to “promote and take a strategic view of integrating AI interventions in dermatology, support dermatology departments to ensure that safe and effective AI interventions are adopted” and provide quality standards for adoption of AI interventions in dermatology care pathways”.
As the Chair, Dr. Matin will oversee various activities including the development of a UK Skin Image Database designed to train, test, and validate domain-specific algorithms, identify and define clinical needs that AI methodologies can potentially address, setting standards to develop and deploy AI interventions in dermatology as well as giving patients a voice in AI. “We believe that AI should address unmet needs, offer better disease management and quality of care, and enhance patient experiences without compromising on their safety,” adds Dr. Matin. “Those who are interested in leveraging AI should also explain what problems they are using AI to solve and why AI is the solution.”