Abstract

Info

Title:: Cell nucleus segmentation in pathological images based on text annotations and Transformer

Author(s):: CHEN Jinling1; CHEN Yu1; TANG Zhuowei2; WEI Jihong2; KE Qi2; JI Yuzhu2; GAO Ziqing2; 1. School of Electrical Engineering and Information, Southwest Petroleum University, Chengdu 610500, China 2. Mianyang Hospital Affiliated to Medical School of University of Electronic Science and Technology of China, Mianyang Central Hospital, Mianyang 621000, China

Keywords:: Keywords: pathological image cell nucleus segmentation text annotation feature fusion

Abstract:: Abstract: A VLi-net based cell nucleus segmentation method integrating convolutional neural networks (CNN) and Vision Transformer (ViT) is proposed to address the limitation that the U-Net with CNN as its backbone is only proficient in capturing local features and has a restricted receptive field. Firstly, to mitigate challenges such as high cost of data annotation and insufficient annotated data, text annotations are introduced to enhance the networks understanding of image information. Secondly, to improve the segmentation performance of VLi-net, ViT and CNN are combined to fully extract global and local features, with multi-receptive field convolution features incorporating into the ViT structure for effectively mitigating the issues of limited local information interaction and single feature representation in ViT. Finally, an interactive fusion module (ViFusion) is used to efficiently fuse the multi-level features from the CNN and ViT branches. Experimental results show that VLi-net achieves a Dice coefficient of 80.85% and a mean intersection over union (MIoU) of 66.83% on the MoNuSeg dataset, obtains a Dice coefficient of 80.53% and a MIoU of 67.54% on the DSB-2018 dataset, and has a Dice coefficient of 86.87% and a MIoU of 77.44% on the TNBC dataset. These findings confirm that VLi-net outperforms other methods across multiple experimental metrics.