Medical images like radiology and pathology images are widely-used in disease diagnosis and treatment [1]}. Given a radiology image, radiologists first examine both the normal and abnormal regions and then use the learned medical knowledge and accumulated working experience to write a coherent report to note down the findings [2]}, [3]}, [4]}. Given the large volume of radiology images, writing reports become a heavy burden for radiologists. Furthermore, for less experienced radiologists, some abnormalities in radiology images may be ignored and consequently not included in the reports [5]}. To relieve radiologists from such heavy workload and remind inexperienced radiologists of abnormalities, automatically generating radiology reports becomes a critical task in clinical practice.
