[1]潘依乐,高永彬.多模态弱监督学习在肝癌图像生成与分割中的应用[J].中国医学物理学杂志,2024,41(1):8-17.[doi:DOI:10.3969/j.issn.1005-202X.2024.01.002]
　PAN Yile,GAO Yongbin.Application of multimodal weakly-supervised learning in image synthesis and segmentation of liver cancer[J].Chinese Journal of Medical Physics,2024,41(1):8-17.[doi:DOI:10.3969/j.issn.1005-202X.2024.01.002]
点击复制

多模态弱监督学习在肝癌图像生成与分割中的应用()
分享到：

《中国医学物理学杂志》[ISSN:1005-202X/CN:44-1351/R]

卷:
41卷
期数:
2024年第1期
页码:
8-17
栏目:
医学影像物理
出版日期:
2024-01-23

文章信息/Info

Title:
Application of multimodal weakly-supervised learning in image synthesis and segmentation of liver cancer
文章编号:
1005-202X（2024）01-0008-10
作者:
潘依乐; 高永彬
上海工程技术大学电子电气工程学院， 上海 201620
Author(s):
PAN Yile;  GAO Yongbin
School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai 201620, China
关键词:
医学图像处理; 模态转换; 肝脏肿瘤分割; 生成式对抗网络; 混合注意力机制
Keywords:
Keywords: medical image processing mode conversion liver tumor segmentation generative adversarial network mixed attention mechanism
分类号:
R318；TP391
DOI:
DOI:10.3969/j.issn.1005-202X.2024.01.002
文献标志码:
A
摘要:
针对MRI软组织对比度高但不是胸部成像护理标准，导致难以获得足够多专家标注的MRI数据的问题，通常将CT转换成MRI图像。由于难以获取对应模态的CT和MRI图像，结合生成式对抗网络的结构特点，提出CSCGAN生成网络模型。该模型以CycleGAN作为框架，由于CycleGAN可能存在模式坍塌问题，同时StyleGAN2能够控制合成图像的样式和特征的细节，实现高分辨率图像的合成，因此将其融入到CycleGAN中，重构了网络的生成器。同时为了减少外部干扰，引入了噪声模块，另外为了防止肿瘤在转换时丢失，修改了网络的鉴别器结构，并加入了混合注意力机制。实验结果显示，与文中其他方法相比，该模型生成的样本图像在Dice相似系数、Hausdorff距离、体积比和平均交并比各项指标上均有所提升，该方法有效实现了肝脏肿瘤病变图像的模态转换，生成的数据能够提高分割网络的准确性。
Abstract:
Although it has high resolution for soft tissues, magnetic resonance imaging (MRI) is not the standard for chest imaging, which results in an insufficient amount of expert-annotated MRI data. Therefore, CT image is usually converted into MRI image. To overcome the difficulty of obtaining the corresponding modal CT and MRI images, a CSCGAN model with CycleGAN as the framework is proposed based on the structural characteristics of generative adversarial networks. Considering the possibility of mode collapse in CycleGAN, StyleGan2 which can control the style and feature details of the synthetic image and realize the synthesis of high-resolution images is integrated into CycleGAN for reconstructing the generator. A noise module is introduced to reduce external interference. In addition, in order to prevent the loss of tumors during conversion, the discriminator structure of the network is modified, and a mixed attention mechanism is added. Experimental results show that compared with the images generated by other methods, those generated by the proposed model are improved in Dice similarity coefficient, Hausdorff distance, volume ratio and mean intersection over union, indicating that the proposed method can effectively realize the mode conversion of liver tumor images, and that the generated data can improve the segmentation accuracy.

相似文献/References:

[1]许红玉,聂生东,刘颖,等.《医学图像处理》课程网站建设[J].中国医学物理学杂志,2013,30(02):4076.[doi:10.3969/j.issn.1005-202X.2013.02.027]
[2]王弈,李传富.人工智能方法在医学图像处理中的研究新进展[J].中国医学物理学杂志,2013,30(03):4138.[doi:10.3969/j.issn.1005-202X.2013.03.013]
[3]张红军,闫士举.基于双侧乳腺图像“共用”阈值分割的乳腺癌近期发病预测[J].中国医学物理学杂志,2017,34(8):820.[doi:DOI:10.3969/j.issn.1005-202X.2017.08.013]
[4]皮一飞,吴茜,裴曦,等.基于掩膜优化的多模态医学图像刚性配准[J].中国医学物理学杂志,2018,35(9):1022.[doi:10.3969/j.issn.1005-202X.2018.09.006]
　PI Yifei,WU Qian,PEI Xi,et al.Rigid registration of multimodal medical images based on mask optimization[J].Chinese Journal of Medical Physics,2018,35(1):1022.[doi:10.3969/j.issn.1005-202X.2018.09.006]
[5]孙旭阳,石更强.基于CT影像的人体踝关节三维重建[J].中国医学物理学杂志,2022,39(7):850.[doi:DOI:10.3969/j.issn.1005-202X.2022.07.010]
　SUN Xuyang,SHI Gengqiang.Three-dimensional reconstruction of the ankle joint based on CT image[J].Chinese Journal of Medical Physics,2022,39(1):850.[doi:DOI:10.3969/j.issn.1005-202X.2022.07.010]
[6]刘燕茹,毕宇越,汪宇航,等.“新医科”背景下医学图像处理教学软件的研究与开发[J].中国医学物理学杂志,2024,41(3):333.[doi:DOI:10.3969/j.issn.1005-202X.2024.03.011]
　LIU Yanru,BI Yuyue,WANG Yuhang,et al.Research and development of teaching software for medical image processing under the background of "new medical science"[J].Chinese Journal of Medical Physics,2024,41(1):333.[doi:DOI:10.3969/j.issn.1005-202X.2024.03.011]
[7]贺斌,高永彬.基于非局部卷积和卷积注意力模块的眩晕眼震诊断方法[J].中国医学物理学杂志,2024,41(5):571.[doi:DOI:10.3969/j.issn.1005-202X.2024.05.007]
　HE Bin,GAO Yongbin.Diagnostic methods for nystagmus in vertigo based on non-local convolution and convolutional block attention module[J].Chinese Journal of Medical Physics,2024,41(1):571.[doi:DOI:10.3969/j.issn.1005-202X.2024.05.007]
[8]王卓然,方志军,王海玲,等.基于双重注意力机制增强的复合型眼震分类框架[J].中国医学物理学杂志,2024,41(9):1093.[doi:DOI:10.3969/j.issn.1005-202X.2024.09.006]
　WANG Zhuoran,FANG Zhijun,WANG Hailing,et al.Composite nystagmus classification framework enhanced by dual attention mechanism[J].Chinese Journal of Medical Physics,2024,41(1):1093.[doi:DOI:10.3969/j.issn.1005-202X.2024.09.006]

备注/Memo

备注/Memo:
【收稿日期】2023-07-09 【基金项目】国家工业信息部和卫健委5G医疗示范项目；广州市科技项目（202206010093）；广东省重点研发项目（2020B010165004） 【作者简介】潘依乐，硕士研究生，研究方向：电子信息，E-mail: panyile@126.com 【通信作者】高永彬，博士，副教授，硕士生导师，研究方向：视觉SLAM技术、三维视觉分析、自然语言处理、知识图谱、无人机/无人车、智能安防，智慧医疗、智能制造，E-mail: gaoyongbin@sues.edu.cn
更新日期/Last Update: 2024-01-23