Pragmatic Radiology Report Generation

Dang Nguyen, Chacha Chen, He He, Chenhao Tan

Research output: Contribution to journalConference articlepeer-review


When pneumonia is not found on a chest X-ray, should the report describe this negative observation or omit it? We argue that this question cannot be answered from the X-ray alone and requires a pragmatic perspective, which captures the communicative goal that radiology reports serve between radiologists and patients. However, the standard image-to-text formulation for radiology report generation fails to incorporate such pragmatic intents. Following this pragmatic perspective, we demonstrate that the indication, which describes why a patient comes for an X-ray, drives the mentions of negative observations. We thus introduce indications as additional input to report generation. With respect to the output, we develop a framework to identify uninferable information from the image, which could be a source of model hallucinations, and limit them by cleaning groundtruth reports. Finally, we use indications and cleaned groundtruth reports to develop pragmatic models, and show that they outperform existing methods not only in new pragmatics-inspired metrics (e.g., +4.3 Negative F1) but also in standard metrics (e.g., +6.3 Positive F1 and +11.0 BLEU-2).

Original languageEnglish (US)
Pages (from-to)385-402
Number of pages18
JournalProceedings of Machine Learning Research
StatePublished - 2023
Event3rd Machine Learning for Health Symposium, ML4H 2023 - New Orleans, United States
Duration: Dec 10 2023 → …

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Control and Systems Engineering
  • Statistics and Probability


Dive into the research topics of 'Pragmatic Radiology Report Generation'. Together they form a unique fingerprint.

Cite this