AI as good as the average radiologist in identifying breast cancer

“This is the first independent comparison conducted to assess the accuracy of several different AI algorithms,” says study author Fredrik Strand, a researcher at the Department of Oncology-Pathology at Karolinska Institutet and a radiologist at Karolinska University Hospital. “We can demonstrate that one of the three algorithms is significantly better than the others and that it equals the accuracy of the average radiologist.”

There are currently a large number of suppliers developing various AI-based medical imaging solutions. This study compares the ability of three of these to identify breast cancer in already taken mammograms. 

The study included mammograms of 8,805 women between the ages of 40 and 74, all of whom had undergone breast cancer screening between 2008 and 2015. Of these, 739 had been diagnosed with breast cancer, either at the time of screening or within the following 12 months. The most successful of the three AI algorithms diagnosed the same percentage of women with cancer as the average radiologist. 

One AI algorithm better than the others

The results also demonstrated that one of the AI algorithms was significantly better than the others.

“We conducted the study in order to find out how far the algorithms have developed and whether there is any difference between the available systems,” Fredrik Strand explains. “The results show that, in principle, the best algorithm is ready for use and that there is a significant difference between the various algorithms on the market.”

Another study of the same research group, recently published in The Lancet Digital Health, showed that an AI algorithm could sort mammography images to indicate which required additional attention by radiologists and which were easily assessed by AI alone without missing any cancer that would otherwise have been detected by a radiologist.

Prospective clinical study in the works

The researchers are now moving on to study how AI can play a part in improving breast cancer screening. Current practice involves two radiologists examining every mammogram. If either of these identify a suspected tumour, the examination proceeds to a discussion involving at least two radiologists who together decide whether the woman should be recalled for further examination. 

“We are planning a prospective clinical study to see how AI works as an independent reviewer of mammograms in a day-to-day clinical environment, both by acting as a third reviewer and by helping to select women who can be offered complementary MRI scans in order to detect cancers at an earlier stage,” says Fredrik Strand. 

The research was conducted at Karolinska Institutet and financed by Region Stockholm. The suppliers of the three algorithms requested anonymity and had no influence over how the study was conducted or how the results were interpreted. 

Publication

”External evaluation of three commercial artificial intelligence algorithms for independent assessment of screening mammograms,” Mattie Salim, Erik Wåhlin, Karin Dembrower, Edward Azavedo, Theodoros Foukakis, Yue Liu, Kevin Smith, Martin Eklund och Fredrik Strand, JAMA Oncology, 27 augusti 2020, doi: 10.1001/jamaoncol.2020.3321