Abstract: |
Chest X-rays (CXRs) are the most frequently performed imaging examinations in clinical settings. Recent advancements in Large Multimodal Models (LMMs) have enabled automated CXR interpretation, enhancing diagnostic accuracy and efficiency. However, despite their strong visual understanding, current Medical LMMs (MLMMs) still face two major challenges: (1) Insufficient region-level understanding and interaction, and (2) Limited accuracy and interpretability due to single-step reasoning. In this paper, we empower MLMMs with anatomy-centric reasoning capabilities to enhance their interactivity and explainability. Specifically, we first propose an Anatomical Ontology-Guided Reasoning (AOR) framework, which centers on cross-modal region-level information to facilitate multi-step reasoning. Next, under the guidance of expert physicians, we develop AOR-Instruction, a large instruction dataset for MLMMs training. Our experiments demonstrate AOR's superior performance in both VQA and report generation tasks. |
Biography: |
| Prof. Emma Shujun Wang is an Assistant Professor at the Department of Biomedical Engineering at The Hong Kong Polytechnic University (PolyU), with affiliations to the Research Institute for Smart Ageing (RISA) and the Research Institute for Artificial Intelligence of Things (RIAIoT). Her research focuses on AI-driven computational methods for precision medicine, particularly in medical image analysis, deep learning, and multimodal learning for disease diagnosis and treatment. She has published over 30 papers in top journals and conferences, such as The Lancet Digital Health and IEEE-TMI, and has received several awards, including Best Paper at MICCAI 2022. Prof. Wang has also contributed to academic leadership, organizing workshops and serving as a distinguished reviewer. |