Enhanced radiology report: Leveraging image enhancement and multi-label transfer learning with attention-based text generation

Abstract

• Automated radiology report generation enhances diagnostic accuracy and reduces radiologist workload. • Novel multi-label learning model predicts 180 abnormality tags with high accuracy. • Transformer-based architecture with GPT-2 and BERT improves report generation in clinical accuracy and context coherence. • Image enhancement able to improves model performance and image clarity. • Ensemble of CNNs boosts multi-label classification and abnormality tag prediction. • Transfer learning with MIMIC-CXR and IU datasets enhances visual feature extraction. Current research in radiology report generation tend to overlook the utilization of abnormalities depicted in medical images. This study introduces a novel radiology report generator that integrates a multi-label learning approach for predicting abnormality tags and employs transformer models for generating reports. Additionally, the research explores contrast-based image enhancement to mitigate noise in medical images, evaluating its impact on model performance. The multi-label learning is trained on a dataset with 180 abnormality labels and the features used as initial weights for MIMIC-CXR, as a visual feature extractor.Imbalance handling and ensemble methods are employed to optimize multi-label model performance for abnormality tag prediction. Multi-head attention, in conjunction with GPT-2, facilitates context building for medical report generation, utilizing BERT embeddings for text feature extraction. Evaluation metrics demonstrate that the proposed model achieves superior performance in both multi-label prediction accuracy 77% and text generation, showing an increase in similarity 28% in average compared to the baseline model. These findings suggest that leveraging transfer learning with an ensemble classifier, combined with a transformer for context building and decoding, effectively utilizes visual and text features. Furthermore, the incorporation of image enhancement techniques significantly impacts model performance.

Ähnliche Arbeiten