Abstract

Info

Title:: Segmentation of rectal cancer lesions on CT and MRI based on cross attention

Author(s):: DENG Jiefu1; XI Zhenghao1; HUANG Chen2; LIU Xiang1; 1. School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai 201620, China 2. Department of General Surgery, Shanghai General Hospital, Shanghai Jiao Tong University, Shanghai 200080, China

Keywords:: Keywords: rectal cancer semantic segmentation cross attention Transformer

Abstract:: Abstract: In response to the limitation of some medical image segmentation models for rectal cancer auxiliary diagnosis that are only applicable to single-modality images, a medical image segmentation method based on a cross attention mechanism that is applicable to both CT and MRI modalities is presented. Considering the different feature representations of CT and MRI images, a cross attention mechanism is proposed to unify the feature representations of the two types of images. In view of the small lesions on rectal cancer images, an improved Swin Transformer segmentation network with 3 branches is established, and the cross attention mechanism is incorporated into it, enabling the model to segment lesion areas in both types of images. The proposed method is validated using CT and MRI image data from patients with rectal cancer. Compared with ADDA, CycleGAN, and SIFA methods, the proposed method improves the accuracy by 2.94%, 3.05%, 0.71% on CT images, and 3.31%, 4.55%, 1.76% on MRI images, respectively, demonstrating its superior segmentation performance for both types of images.