AI is as good as pathologists at diagnosing Celiac disease, study finds

The AI tool, which has been trained on almost 3,400 scanned biopsies from four NHS hospitals, could speed up diagnosis of the condition and take pressure off stretched healthcare resources, as well as improving diagnosis in developing nations, where shortages of pathologists are severe.

Digital tools that can speed up or even automate analysis of diagnostic tests are beginning to show real promise for reducing the demands on pathologists. A large amount of this work has focused on the detection of cancer, but researchers are beginning to look at opportunities to diagnose other types of disease.

One condition being looked at by scientists at the University of Cambridge is coeliac disease, an autoimmune disease trigged by consuming gluten. It causes symptoms that include stomach cramps, diarrhoea, skin rashes, weight loss, fatigue and anaemia. Because symptoms vary so much between individuals, patients often have difficulty in receiving an accurate diagnosis.

The gold standard for diagnosing coeliac disease is via a biopsy of the duodenum (part of the small intestine). Pathologists will then analyse the sample under a microscope or on a computer to look for damage to the villi, tiny hair-like projections that line the inside of the small intestine.

Interpreting biopsies, which often have subtle changes, can be subjective. Pathologists use a classification system known as the Marsh-Oberhuber scale to judge the severity of a case, ranging from zero (the villi are normal and the patient is unlikely to have the disease) to four (the villi are completely flattened).

In research published today in the New England Journal of Medicine AI, Cambridge researchers developed a machine learning algorithm to classify biopsy image data. The algorithm was trained and tested on a large-scale, diverse dataset consisting of over 4,000 images obtained from five different hospitals using five different scanners from four different companies.

Senior author Professor Elizabeth Soilleux from the Department of Pathology and Churchill College, University of Cambridge, said: “Coeliac disease affects as many as one in 100 people and can cause serious illness, but getting a diagnosis is not straightforward. It can take many years to receive an accurate diagnosis, and at a time of intense pressures on healthcare systems, these delays are likely to continue. AI has the potential to speed up this process, allowing patients to receive a diagnosis faster, while at the same time taking pressure off NHS waiting lists.”

The team tested their algorithm on an independent data set of almost 650 images from a previously unseen source. Based on comparisons with the original pathologists’ diagnoses, the researchers showed that the model was correct in its diagnosis in more than 97 cases out of 100.

The model had a sensitivity of over 95% — meaning that it correctly identified more than 95 cases out of 100 individuals who had coeliac disease. It also had a specificity of almost 98% — meaning that it correctly identified in nearly 98 cases out of 100 individuals who did not have coeliac disease.

Previous research by the team has shown that even pathologists can disagree on diagnoses. When shown a series of 100 slides and asked to diagnose whether a patient had coeliac disease, did not have the disease, or whether the diagnosis was indeterminate, the team showed that there was disagreement in more than one in five cases.

This time round, the researchers asked four pathologists to review 30 slides and found that a pathologist was as likely to agree with the AI model as they were with a second pathologist.

Dr Florian Jaeckle, also from the Department of Pathology, and a Research Fellow at Hughes Hall, Cambridge, said: “This is the first time AI has been shown to diagnose as accurately as an experienced pathologist whether an individual has coeliac or not. Because we trained it on data sets generated under a number of different conditions, we know that it should be able to work in a wide range of settings, where biopsies are processed and imaged differently.

“This is an important step towards speeding up diagnoses and freeing up pathologists’ time to focus on more complex or urgent cases. Our next step is to test the algorithm in a much larger clinical sample, putting us in a position to share this device with the regulator, bringing us nearer to this tool being used in the NHS.”

The researchers have been working with patient groups, including through Coeliac UK, to share their approach and discuss with them their receptiveness to technology such as this being used.

“When we speak to patients, they are generally very receptive to the use of AI for diagnosing coeliac disease,” added Dr Jaeckle. “This no doubt partly reflects their experiences of the difficulties and delays in receiving a diagnosis.

“One issue that comes up frequently with both patients and clinicians is the issue of ‘explainability’ — being able to understand and explain how AI reaches its diagnosis. It’s important for us as researchers and for regulators to bear this mind if we want to ensure there is public trust in applications of AI in medicine.”

Professor Soilleux is a consultant haematopathologist at Cambridge University Hospitals NHS Foundation Trust. Together with Dr Jaeckle, she has set up a spinout company, Lyzeum Ltd, to commercialise the algorithm.

The research was funded by Coeliac UK, Innovate UK, the Cambridge Centre for Data-Driven Discovery and the National Institute for Health and Care Research.

Keira Shepherd, Research Officer at Coeliac UK, said: “During the diagnostic process, it’s vital that patients keep gluten in their diet to ensure that the diagnosis is accurate. But this can cause uncomfortable symptoms. That’s why it’s really important that they are able to receive an accurate diagnosis as quickly as possible.

“This research demonstrates one potential way to speed up part of the diagnosis journey. At Coeliac UK, we’re proud to have funded the early stages of this work, which initially focused on training a system to differentiate between healthy control biopsies and biopsies of patients with coeliac disease. We hope that one day this technology will be used to help patients receive a quick and accurate diagnosis.”