Abstract

Info

Title:: Incomplete multimodality MR image synthesis based on generative adversarial network

Author(s):: XU Panpan; ZHANG Dong; YUAN Dalong; School of Physics and Technology, Wuhan University, Wuhan 430072, China

Keywords:: Keywords: magnetic resonance imaging generative adversarial network hybrid attention module multimodal synthesis

Abstract:: Abstract: An incomplete modality synthesis method for multimodality magnetic resonance (MR) images based on generative adversarial network (GAN) is proposed to predict the complete modal sequence in different scenarios. The proposed model uses GAN which consists of generator and discriminator as the backbone of the network. The generator contains feature extraction, feature fusion and image generation. U-Net is used for encoding during feature extraction, and the features of the 4 modalities are fused before being input into the hybrid attention module, so that the network can adaptively adjust the weights for different modalities and finally generate images. The discriminator consists of 4 same discriminators which network structure is improved from PatchGan to discriminate different modalities. Compared with other state-of-the-art modality synthesis algorithms, the proposed method achieves better results in both visual effects and objective criteria, with peak signal-to-noise ratios higher than 23 dB and structural similarity index measurements above 0.99.