The algorithms were developed as part of a global competition, the Prostate cANcer graDe Assessment (PANDA) challenge, that drew AI experts from 65 countries. The full development set of 10,616 digitized biopsies is publicly available for noncommercial research use at https://panda.grand-challenge.org/.
As reported in Nature Medicine, Dr. Kimmo Kartasalo and Dr. Martin Eklund, both of the Karolinska Institutet, and colleagues organized the PANDA challenge by compiling and releasing the digitized biopsies from Europe for AI development. They then reproduced the top-performing algorithms and externally validated their generalization to independent U.S. and EU cohorts and compared them with reviews by pathologists.
During the competition phase, 1,010 teams consisting of 1,290 developers participated and submitted at least one algorithm. The algorithms were blindly validated simultaneously on the internal validation set. On U.S. and European external validation sets, the algorithms achieved agreements of 0.862 and 0.868 (quadratically weighted kappa) with expert uropathologists.
The authors conclude, "Successful generalization across different patient populations, laboratories and reference standards, achieved by a variety of algorithmic approaches, warrants evaluating AI-based Gleason grading in prospective clinical trials."
Drs. Kartasalo and Eklund told Reuters Health by email, "Now that it has been shown that AI algorithms for this task can successfully generalize across international patient cohorts and samples originating from different hospitals, the next step is to carefully evaluate how to best integrate such algorithms in the clinical workflow, and to measure their effect prospectively in routine use. This requires setting up clinical trials."
"If the results of the trials confirm the benefits of the technology in an everyday setting, we believe widespread use is likely to follow," they said. "An AI tool could be implemented as part of the clinical routine in various ways...Most likely it would best be used routinely to process the bulk of the samples, while highlighting the most difficult-to-diagnose cases and anything out of the ordinary for the medical expert to focus on."
"In some scenarios," they noted, "full AI-based automation could be a reasonable option, if the alternative is having no pathology assessment at all, such as in some regions in developing countries."
"There are risks involved with algorithms developed using data from a particular hospital and country, which may perform poorly and unexpectedly when applied at different sites," they cautioned. "The quality of proposed AI systems in this respect varies considerably. However, our latest study, for example, shows that these problems are solvable with a rigorous study design and careful validation."
Dr. Sophia Kamran, a radiation oncologist at the Massachusetts General Hospital Cancer Center in Boston, commented on the study in an email to Reuters Health. "There are some biases and limitations to the AI algorithms and how they were developed - for example, the AI algorithm development was based on single individual biopsies, whereas in the clinic, Gleason grading occurs on a patient-level, using multiple cores/biopsies from a single individual."
"AI algorithms for Gleason grading need to be incorporated and validated in prospective clinical trials, but AI can be a useful tool to reduce the significant variability between different pathologists, and it can be particularly useful for general pathologists, who have been found to undergrade prostate cancer," she noted.
"Competitive crowd-sourcing methodology can lead to successful development of high-performing medical algorithms," she affirmed. "This methodology should be reproduced for future medical AI development to rapidly create additional high-performing algorithms for incorporation and evaluation in the clinic."
Several authors have patents related to prostate cancer diagnostics or machine learning for medical images. Ten coauthors are employees of Google LLC and own Alphabet stock, and several coauthors received fees from the company. Five coauthors are employees of VUNO Inc.
SOURCE: https://go.nature.com/3nTRYKY Nature Medicine, online January 13, 2022.
By Marilynn Larkin
Posted on
site created by:
© 2024 Medicom Medical Publishers. All rights reserved. Terms and Conditions | Privacy Policy