Abstract

Info

Title:: Application of multimodal weakly-supervised learning in image synthesis and segmentation of liver cancer

Author(s):: PAN Yile; GAO Yongbin; School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai 201620, China

Keywords:: Keywords: medical image processing mode conversion liver tumor segmentation generative adversarial network mixed attention mechanism

Abstract:: Although it has high resolution for soft tissues, magnetic resonance imaging (MRI) is not the standard for chest imaging, which results in an insufficient amount of expert-annotated MRI data. Therefore, CT image is usually converted into MRI image. To overcome the difficulty of obtaining the corresponding modal CT and MRI images, a CSCGAN model with CycleGAN as the framework is proposed based on the structural characteristics of generative adversarial networks. Considering the possibility of mode collapse in CycleGAN, StyleGan2 which can control the style and feature details of the synthetic image and realize the synthesis of high-resolution images is integrated into CycleGAN for reconstructing the generator. A noise module is introduced to reduce external interference. In addition, in order to prevent the loss of tumors during conversion, the discriminator structure of the network is modified, and a mixed attention mechanism is added. Experimental results show that compared with the images generated by other methods, those generated by the proposed model are improved in Dice similarity coefficient, Hausdorff distance, volume ratio and mean intersection over union, indicating that the proposed method can effectively realize the mode conversion of liver tumor images, and that the generated data can improve the segmentation accuracy.