Artificial intelligence system reduces false-positive findings in the interpretation of breast ultrasound exams

Yiqiu Shen; Farah E. Shamout; Jamie R. Oliver; Jan Witowski; Kawshik Kannan; Jungkyu Park; Nan Wu; Connor Huddleston; Stacey Wolfson; Alexandra Millet; Robin Ehrenpreis; Divya Awal; Cathy Tyma; Naziya Samreen; Yiming Gao; Chloe Chhor; Stacey Gandhi; Cindy Lee; Sheila Kumari-Subaiya; Cindy Leonard; Reyhan Mohammed; Christopher Moczulski; Jaime Altabet; James Babb; Alana Lewin; Beatriu Reig; Linda Moy; Laura Heacock; Krzysztof J. Geras

doi:10.1038/s41467-021-26023-2

Artificial intelligence system reduces false-positive findings in the interpretation of breast ultrasound exams

Yiqiu Shen, Farah E. Shamout, Jamie R. Oliver, Jan Witowski, Kawshik Kannan, Jungkyu Park, Nan Wu, Connor Huddleston, Stacey Wolfson, Alexandra Millet, Robin Ehrenpreis, Divya Awal, Cathy Tyma, Naziya Samreen, Yiming Gao, Chloe Chhor, Stacey Gandhi, Cindy Lee, Sheila Kumari-Subaiya, Cindy LeonardReyhan Mohammed, Christopher Moczulski, Jaime Altabet, James Babb, Alana Lewin, Beatriu Reig, Linda Moy, Laura Heacock, Krzysztof J. Geras

Computer Engineering

Research output: Contribution to journal › Article › peer-review

Abstract

Though consistently shown to detect mammographically occult cancers, breast ultrasound has been noted to have high false-positive rates. In this work, we present an AI system that achieves radiologist-level accuracy in identifying breast cancer in ultrasound images. Developed on 288,767 exams, consisting of 5,442,907 B-mode and Color Doppler images, the AI achieves an area under the receiver operating characteristic curve (AUROC) of 0.976 on a test set consisting of 44,755 exams. In a retrospective reader study, the AI achieves a higher AUROC than the average of ten board-certified breast radiologists (AUROC: 0.962 AI, 0.924 ± 0.02 radiologists). With the help of the AI, radiologists decrease their false positive rates by 37.3% and reduce requested biopsies by 27.8%, while maintaining the same level of sensitivity. This highlights the potential of AI in improving the accuracy, consistency, and efficiency of breast ultrasound diagnosis.

Original language	English (US)
Article number	5645
Journal	Nature communications
Volume	12
Issue number	1
DOIs	https://doi.org/10.1038/s41467-021-26023-2
State	Published - Dec 1 2021

ASJC Scopus subject areas

General Chemistry
General Biochemistry, Genetics and Molecular Biology
General Physics and Astronomy

Access to Document

10.1038/s41467-021-26023-2

Cite this

Shen, Y., Shamout, F. E., Oliver, J. R., Witowski, J., Kannan, K., Park, J., Wu, N., Huddleston, C., Wolfson, S., Millet, A., Ehrenpreis, R., Awal, D., Tyma, C., Samreen, N., Gao, Y., Chhor, C., Gandhi, S., Lee, C., Kumari-Subaiya, S., ... Geras, K. J. (2021). Artificial intelligence system reduces false-positive findings in the interpretation of breast ultrasound exams. Nature communications, 12(1), Article 5645. https://doi.org/10.1038/s41467-021-26023-2

Shen, Y, Shamout, FE, Oliver, JR, Witowski, J, Kannan, K, Park, J, Wu, N, Huddleston, C, Wolfson, S, Millet, A, Ehrenpreis, R, Awal, D, Tyma, C, Samreen, N, Gao, Y, Chhor, C, Gandhi, S, Lee, C, Kumari-Subaiya, S, Leonard, C, Mohammed, R, Moczulski, C, Altabet, J, Babb, J, Lewin, A, Reig, B, Moy, L, Heacock, L & Geras, KJ 2021, 'Artificial intelligence system reduces false-positive findings in the interpretation of breast ultrasound exams', Nature communications, vol. 12, no. 1, 5645. https://doi.org/10.1038/s41467-021-26023-2

@article{116d25c557c14391b794d7bf8a2a4e57,

title = "Artificial intelligence system reduces false-positive findings in the interpretation of breast ultrasound exams",

abstract = "Though consistently shown to detect mammographically occult cancers, breast ultrasound has been noted to have high false-positive rates. In this work, we present an AI system that achieves radiologist-level accuracy in identifying breast cancer in ultrasound images. Developed on 288,767 exams, consisting of 5,442,907 B-mode and Color Doppler images, the AI achieves an area under the receiver operating characteristic curve (AUROC) of 0.976 on a test set consisting of 44,755 exams. In a retrospective reader study, the AI achieves a higher AUROC than the average of ten board-certified breast radiologists (AUROC: 0.962 AI, 0.924 ± 0.02 radiologists). With the help of the AI, radiologists decrease their false positive rates by 37.3% and reduce requested biopsies by 27.8%, while maintaining the same level of sensitivity. This highlights the potential of AI in improving the accuracy, consistency, and efficiency of breast ultrasound diagnosis.",

author = "Yiqiu Shen and Shamout, {Farah E.} and Oliver, {Jamie R.} and Jan Witowski and Kawshik Kannan and Jungkyu Park and Nan Wu and Connor Huddleston and Stacey Wolfson and Alexandra Millet and Robin Ehrenpreis and Divya Awal and Cathy Tyma and Naziya Samreen and Yiming Gao and Chloe Chhor and Stacey Gandhi and Cindy Lee and Sheila Kumari-Subaiya and Cindy Leonard and Reyhan Mohammed and Christopher Moczulski and Jaime Altabet and James Babb and Alana Lewin and Beatriu Reig and Linda Moy and Laura Heacock and Geras, {Krzysztof J.}",

note = "Publisher Copyright: {\textcopyright} 2021, The Author(s).",

year = "2021",

month = dec,

day = "1",

doi = "10.1038/s41467-021-26023-2",

language = "English (US)",

volume = "12",

journal = "Nature communications",

issn = "2041-1723",

publisher = "Nature Publishing Group",

number = "1",

}

TY - JOUR

T1 - Artificial intelligence system reduces false-positive findings in the interpretation of breast ultrasound exams

AU - Shen, Yiqiu

AU - Shamout, Farah E.

AU - Oliver, Jamie R.

AU - Witowski, Jan

AU - Kannan, Kawshik

AU - Park, Jungkyu

AU - Wu, Nan

AU - Huddleston, Connor

AU - Wolfson, Stacey

AU - Millet, Alexandra

AU - Ehrenpreis, Robin

AU - Awal, Divya

AU - Tyma, Cathy

AU - Samreen, Naziya

AU - Gao, Yiming

AU - Chhor, Chloe

AU - Gandhi, Stacey

AU - Lee, Cindy

AU - Kumari-Subaiya, Sheila

AU - Leonard, Cindy

AU - Mohammed, Reyhan

AU - Moczulski, Christopher

AU - Altabet, Jaime

AU - Babb, James

AU - Lewin, Alana

AU - Reig, Beatriu

AU - Moy, Linda

AU - Heacock, Laura

AU - Geras, Krzysztof J.

PY - 2021/12/1

Y1 - 2021/12/1

N2 - Though consistently shown to detect mammographically occult cancers, breast ultrasound has been noted to have high false-positive rates. In this work, we present an AI system that achieves radiologist-level accuracy in identifying breast cancer in ultrasound images. Developed on 288,767 exams, consisting of 5,442,907 B-mode and Color Doppler images, the AI achieves an area under the receiver operating characteristic curve (AUROC) of 0.976 on a test set consisting of 44,755 exams. In a retrospective reader study, the AI achieves a higher AUROC than the average of ten board-certified breast radiologists (AUROC: 0.962 AI, 0.924 ± 0.02 radiologists). With the help of the AI, radiologists decrease their false positive rates by 37.3% and reduce requested biopsies by 27.8%, while maintaining the same level of sensitivity. This highlights the potential of AI in improving the accuracy, consistency, and efficiency of breast ultrasound diagnosis.

AB - Though consistently shown to detect mammographically occult cancers, breast ultrasound has been noted to have high false-positive rates. In this work, we present an AI system that achieves radiologist-level accuracy in identifying breast cancer in ultrasound images. Developed on 288,767 exams, consisting of 5,442,907 B-mode and Color Doppler images, the AI achieves an area under the receiver operating characteristic curve (AUROC) of 0.976 on a test set consisting of 44,755 exams. In a retrospective reader study, the AI achieves a higher AUROC than the average of ten board-certified breast radiologists (AUROC: 0.962 AI, 0.924 ± 0.02 radiologists). With the help of the AI, radiologists decrease their false positive rates by 37.3% and reduce requested biopsies by 27.8%, while maintaining the same level of sensitivity. This highlights the potential of AI in improving the accuracy, consistency, and efficiency of breast ultrasound diagnosis.

UR - http://www.scopus.com/inward/record.url?scp=85115675338&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85115675338&partnerID=8YFLogxK

U2 - 10.1038/s41467-021-26023-2

DO - 10.1038/s41467-021-26023-2

M3 - Article

C2 - 34561440

AN - SCOPUS:85115675338

SN - 2041-1723

VL - 12

JO - Nature communications

JF - Nature communications

IS - 1

M1 - 5645

ER -

Artificial intelligence system reduces false-positive findings in the interpretation of breast ultrasound exams

Abstract

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this