Show simple item record

AuthorEhteshami Bejnordi, Babak
AuthorVeta, Mitko
AuthorJohannes van Diest, Paul
Authorvan Ginneken, Bram
AuthorKarssemeijer, Nico
AuthorLitjens, Geert
Authorvan der Laak, Jeroen A W M
Authorthe CAMELYON16 Consortium
AuthorHermsen, Meyke
AuthorManson, Quirine F
AuthorBalkenhol, Maschenka
AuthorGeessink, Oscar
AuthorStathonikos, Nikolaos
Authorvan Dijk, Marcory Crf
AuthorBult, Peter
AuthorBeca, Francisco
AuthorBeck, Andrew H
AuthorWang, Dayong
AuthorKhosla, Aditya
AuthorGargeya, Rishab
AuthorIrshad, Humayun
AuthorZhong, Aoxiao
AuthorDou, Qi
AuthorLi, Quanzheng
AuthorChen, Hao
AuthorLin, Huang-Jing
AuthorHeng, Pheng-Ann
AuthorHaß, Christian
AuthorBruni, Elia
AuthorWong, Quincy
AuthorHalici, Ugur
AuthorÖner, Mustafa Ümit
AuthorCetin-Atalay, Rengul
AuthorBerseth, Matt
AuthorKhvatkov, Vitali
AuthorVylegzhanin, Alexei
AuthorKraus, Oren
AuthorShaban, Muhammad
AuthorRajpoot, Nasir
AuthorAwan, Ruqayya
AuthorSirinukunwattana, Korsuk
AuthorQaiser, Talha
AuthorTsang, Yee-Wah
AuthorTellez, David
AuthorAnnuscheit, Jonas
AuthorHufnagl, Peter
AuthorValkonen, Mira
AuthorKartasalo, Kimmo
AuthorLatonen, Leena
AuthorRuusuvuori, Pekka
AuthorLiimatainen, Kaisa
AuthorAlbarqouni, Shadi
AuthorMungal, Bharti
AuthorGeorge, Ami
AuthorDemirci, Stefanie
AuthorNavab, Nassir
AuthorWatanabe, Seiryo
AuthorSeno, Shigeto
AuthorTakenaka, Yoichi
AuthorMatsuda, Hideo
AuthorAhmady Phoulady, Hady
AuthorKovalev, Vassili
AuthorKalinovsky, Alexander
AuthorLiauchuk, Vitali
AuthorBueno, Gloria
AuthorFernandez-Carrobles, M Milagro
AuthorSerrano, Ismael
AuthorDeniz, Oscar
AuthorRacoceanu, Daniel
AuthorVenâncio, Rui
Available date2020-08-20T08:06:22Z
Publication Date2017
Publication NameJAMA - Journal of the American Medical Association
ResourceScopus
ISSN987484
URIhttp://dx.doi.org/10.1001/jama.2017.14585
URIhttp://hdl.handle.net/10576/15710
AbstractIMPORTANCE: Application of deep learning algorithms to whole-slide pathology imagescan potentially improve diagnostic accuracy and efficiency. OBJECTIVE: Assess the performance of automated deep learning algorithms at detecting metastases in hematoxylin and eosin-stained tissue sections of lymph nodes of women with breast cancer and compare it with pathologists' diagnoses in a diagnostic setting. DESIGN, SETTING, AND PARTICIPANTS: Researcher challenge competition (CAMELYON16) to develop automated solutions for detecting lymph node metastases (November 2015-November 2016). A training data set of whole-slide images from 2 centers in the Netherlands with (n = 110) and without (n = 160) nodal metastases verified by immunohistochemical staining were provided to challenge participants to build algorithms. Algorithm performance was evaluated in an independent test set of 129 whole-slide images (49 with and 80 without metastases). The same test set of corresponding glass slides was also evaluated by a panel of 11 pathologists with time constraint (WTC) from the Netherlands to ascertain likelihood of nodal metastases for each slide in a flexible 2-hour session, simulating routine pathology workflow, and by 1 pathologist without time constraint (WOTC). EXPOSURES: Deep learning algorithms submitted as part of a challenge competition or pathologist interpretation. MAIN OUTCOMES AND MEASURES: The presence of specific metastatic foci and the absence vs presence of lymph node metastasis in a slide or image using receiver operating characteristic curve analysis. The 11 pathologists participating in the simulation exercise rated their diagnostic confidence as definitely normal, probably normal, equivocal, probably tumor, or definitely tumor. RESULTS: The area under the receiver operating characteristic curve (AUC) for the algorithms ranged from 0.556 to 0.994. The top-performing algorithm achieved a lesion-level, true-positive fraction comparable with that of the pathologist WOTC (72.4% [95% CI, 64.3%-80.4%]) at a mean of 0.0125 false-positives per normal whole-slide image. For the whole-slide image classification task, the best algorithm (AUC, 0.994 [95% CI, 0.983-0.999]) performed significantly better than the pathologists WTC in a diagnostic simulation (mean AUC, 0.810 [range, 0.738-0.884]; P <.001). The top 5 algorithms had a mean AUC that was comparable with the pathologist interpreting the slides in the absence of time constraints (mean AUC, 0.960 [range, 0.923-0.994] for the top 5 algorithms vs 0.966 [95% CI, 0.927-0.998] for the pathologist WOTC). CONCLUSIONS AND RELEVANCE: In the setting of a challenge competition, some deep learning algorithms achieved better diagnostic performance than a panel of 11 pathologists participating in a simulation exercise designed to mimic routine pathology workflow; algorithm performance was comparable with an expert pathologist interpreting whole-slide images without time constraints. Whether this approach has clinical utility will require evaluation in a clinical setting. 2017 American Medical Association. All rights reserved.
SponsorData collection and annotation were funded by Stichting IT Projecten and by the Fonds Economische Structuurversterking (tEPIS/TRAIT project; LSH-FES Program 2009; DFES1029161 and FES1103JJT8U). Fonds Economische Structuurversterking also supported (in kind) web-access to whole-slide images. This work was supported by grant 601040 from the Seventh Framework Programme for Research-funded VPH-PRISM project of the European Union (Mr Ehteshami Bejnordi).
Languageen
PublisherAmerican Medical Association
SubjectHematoxylin
Cancer Classification
Histopathology
TitleDiagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer
TypeArticle
Pagination2199-2210
Issue Number22
Volume Number318
dc.accessType Open Access


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record