Detailed Annotations of Chest X-Rays via CT Projection for Report Understanding
Constantin Marc Seibold (Karlsruhe Institute of Technology),* Simon Reiß (Karlsruhe Institute of Technology), M. Saquib Sarfraz (Mercdes-Benz / KIT), Matthias A. Fink (University Hospital Heidelberg), Victoria Mayer (Mayer), Jan Sellner (German Cancer Research Center), Moon Sung Kim (University Hospital of Essen), Klaus H. Maier-Hein (German Cancer Research Center (DKFZ)), Jens Kleesiek (Institute for AI in Medicine (IKIM), University Hospital Essen), Rainer Stiefelhagen (Karlsruhe Institute of Technology)
The 33rd British Machine Vision Conference
Abstract
In clinical radiology reports, doctors capture important information about the patient's health status. They convey their observations from raw medical imaging data about the inner structures of a patient. As such, formulating reports requires medical experts to possess wide-ranging knowledge about anatomical regions with their normal, healthy appearance as well as the ability to recognize abnormalities. This explicit grasp on both the patient's anatomy and their appearance is missing in current medical image-processing systems as annotations are especially difficult to gather. This renders the models to be narrow experts e.g. for identifying specific diseases. In this work, we recover this missing link by adding human anatomy into the mix and enable the association of content in medical reports to their occurrence in associated imagery (medical phrase grounding). To exploit anatomical structures in this scenario, we present a sophisticated automatic pipeline to gather and integrate human bodily structures from computed tomography datasets, which we incorporate in our PAXRay: A Projected dataset for the segmentation of Anatomical structures in X-Ray data. Our evaluation shows that methods that take advantage of anatomical information benefit heavily in visually grounding radiologists' findings, as our anatomical segmentations allow for up to absolute 50% better grounding results on the OpenI dataset than commonly used region proposals.Video
Citation
@inproceedings{Seibold_2022_BMVC,
author = {Constantin Marc Seibold and Simon Reiß and M. Saquib Sarfraz and Matthias A. Fink and Victoria Mayer and Jan Sellner and Moon Sung Kim and Klaus H. Maier-Hein and Jens Kleesiek and Rainer Stiefelhagen},
title = {Detailed Annotations of Chest X-Rays via CT Projection for Report Understanding},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0058.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by