[1]周信宏,黄钢.PET-CT多模态融合在图像语义分割的应用进展[J].中国医学物理学杂志,2023,40(6):683-694.[doi:DOI:10.3969/j.issn.1005-202X.2023.06.004]
　ZHOU Xinhong,HUANG Gang.Multimodal fusion of PET-CT for semantic image segmentation: a review[J].Chinese Journal of Medical Physics,2023,40(6):683-694.[doi:DOI:10.3969/j.issn.1005-202X.2023.06.004]
点击复制

PET-CT多模态融合在图像语义分割的应用进展()
分享到：

《中国医学物理学杂志》[ISSN:1005-202X/CN:44-1351/R]

卷:
40卷
期数:
2023年第6期
页码:
683-694
栏目:
医学影像物理
出版日期:
2023-06-27

文章信息/Info

Title:
Multimodal fusion of PET-CT for semantic image segmentation: a review
文章编号:
1005-202X（2023）06-0683-12
作者:
周信宏1; 黄钢2
1.上海理工大学健康科学与工程学院， 上海 200093； 2.上海健康医学院附属嘉定中心医院上海市分子影像学重点实验室， 上海 201318
Author(s):
ZHOU Xinhong1;  HUANG Gang2
1. School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China 2. Shanghai Key Laboratory of Molecular Imaging, Jiading District Central Hospital, Shanghai University of Medicine and Health Sciences, Shanghai 201318, China
关键词:
PET-CT; 卷积神经网络; 语义分割; 多模态; 综述
Keywords:
Keywords: PET-CT convolutional neural network semantic segmentation multimodal review
分类号:
R318；TP391
DOI:
DOI:10.3969/j.issn.1005-202X.2023.06.004
文献标志码:
A
摘要:
近年来，深度学习在各种医学图像语义分割任务中表现出了优异的性能。然而，在癌症的淋巴结转移以及一些细小病灶的分割任务中，基于单模态图像的语义分割仍然存在局限性。PET-CT这一多模态成像模式提供了有关结构和生理病理的解剖和功能信息，被认为是癌症诊断、分期和治疗反应评估的首选成像设备。PET-CT携带的多模态互补信息被引入深度卷积神经网络的语义分割模型中辅助病灶的分割。首先，聚焦于PET-CT多模态图像特征融合在卷积神经网络图像语义分割中的应用，介绍多模态成像的特点以及常用的医学图像分割网络。然后，归纳和总结了现阶段基于深度学习的多模态融合的3种思路，并划分为早期融合、后期融合和混合融合，并从分割的性能、参数量的大小、方法的亮点与不足等维度对多模态融合分割方法进行优缺点分析。最后，对现阶段多模态医学图像分割存在的问题进行讨论。
Abstract:
Abstract: In recent years, deep learning has exhibited excellent performance in various semantic image segmentation tasks. However, the semantic segmentation based on monomodal image still has limitations in the segmentations of cancer with lymph node metastasis and some small lesions. PET-CT which is a multimodal imaging mode, providing anatomical and functional information about structure, physiology and pathology, is considered to be the preferred imaging choice for cancer diagnosis, staging and treatment response evaluation. The multimodal complementary information carried by PET-CT is introduced into the semantic segmentation model of deep convolutional neural network to assist the segmentation of lesions. The study focuses on the application of PET-CT multimodal image feature fusion in semantic image segmentation using convolutional neural network, and introduces the characteristics of multimodal imaging and commonly-used medical image segmentation networks. Three ideas for multimodal fusion based on deep learning at this stage are summarized and divided into early fusion, late fusion and hybrid fusion. The pros and cons of multimodal fusion based segmentation methods are analyzed in terms of segmentation performance, parameter size, method highlights and shortcomings. Finally, the current issues existing in multimodal medical image segmentation are discussed.

相似文献/References:

[1]陶源,王佳飞,杜俊龙,等.基于卷积神经网络的细胞识别[J].中国医学物理学杂志,2017,34(1):53.[doi:10.3969/j.issn.1005-202X.2017.01.011]
　[J].Chinese Journal of Medical Physics,2017,34(6):53.[doi:10.3969/j.issn.1005-202X.2017.01.011]
[2]刘岩,李幼军,陈萌. 基于固有模态分解和深度学习的抑郁症脑电信号分类分析[J].中国医学物理学杂志,2017,34(9):963.[doi:DOI:10.3969/j.issn.1005-202X.2017.09.021]
　[J].Chinese Journal of Medical Physics,2017,34(6):963.[doi:DOI:10.3969/j.issn.1005-202X.2017.09.021]
[3]张俊,朱金汉,庄永东,等. 基于卷积神经网络CT/CBCT影像质量自动分析[J].中国医学物理学杂志,2018,35(5):557.[doi:DOI:10.3969/j.issn.1005-202X.2018.05.012]
　ZHANG Jun,ZHU Jinhan,ZHUANG Yongdong,et al. Automatic analysis of CT/CBCT image quality based on convolutional neural network[J].Chinese Journal of Medical Physics,2018,35(6):557.[doi:DOI:10.3969/j.issn.1005-202X.2018.05.012]
[4]邓金城,彭应林,刘常春,等. 深度卷积神经网络在放射治疗计划图像分割中的应用[J].中国医学物理学杂志,2018,35(6):621.[doi:DOI:10.3969/j.issn.1005-202X.2018.06.001]
　DENG Jincheng,PENG Yinglin,LIU Changchun,et al. Application of deep convolution neural network in radiotherapy planning image segmentation[J].Chinese Journal of Medical Physics,2018,35(6):621.[doi:DOI:10.3969/j.issn.1005-202X.2018.06.001]
[5]申代友,库洪安,皮红英,等. 基于深度相机的老年跌倒监护系统[J].中国医学物理学杂志,2019,36(2):223.[doi:DOI:10.3969/j.issn.1005-202X.2019.02.019]
　SHEN Daiyou,KU Hongan,PI Hongying,et al. Depth camera-based fall detection system for the elderly[J].Chinese Journal of Medical Physics,2019,36(6):223.[doi:DOI:10.3969/j.issn.1005-202X.2019.02.019]
[6]宫进昌,赵尚义,王远军. 基于深度学习的医学图像分割研究进展[J].中国医学物理学杂志,2019,36(4):420.[doi:DOI:10.3969/j.issn.1005-202X.2019.04.010]
　GONG Jinchang,ZHAO Shangyi,WANG Yuanjun.Research progress on deep learning-based medical image segmentation[J].Chinese Journal of Medical Physics,2019,36(6):420.[doi:DOI:10.3969/j.issn.1005-202X.2019.04.010]
[7]王自强,刘洪运,石金龙,等.基于卷积神经网络的心电图心博识别[J].中国医学物理学杂志,2019,36(8):938.[doi:DOI:10.3969/j.issn.1005-202X.2019.08.015]
　WANG Ziqiang,LIU Hongyun,SHI Jinlong,et al.ECG heartbeat recognition based on convolution neural network[J].Chinese Journal of Medical Physics,2019,36(6):938.[doi:DOI:10.3969/j.issn.1005-202X.2019.08.015]
[8]纪春阳,徐秀林,王燕. 深度神经网络技术在肿瘤细胞识别中的应用[J].中国医学物理学杂志,2019,36(9):1113.[doi:DOI:10.3969/j.issn.1005-202X.2019.09.022]
　JI Chunyang,XU Xiulin,WANG Yan. Application of deep neural network in tumor cell recognition[J].Chinese Journal of Medical Physics,2019,36(6):1113.[doi:DOI:10.3969/j.issn.1005-202X.2019.09.022]
[9]徐航,随力,张靖雯,等.卷积神经网络在医学图像分割中的研究进展[J].中国医学物理学杂志,2019,36(11):1302.[doi:DOI:10.3969/j.issn.1005-202X.2019.11.011]
　XU Hang,SUI Li,ZHANG Jingwen,et al.Progress on convolutional neural network in medical image segmentation[J].Chinese Journal of Medical Physics,2019,36(6):1302.[doi:DOI:10.3969/j.issn.1005-202X.2019.11.011]
[10]蒋家良,罗勇,何奕松,等.特征区域再聚焦提升全卷积神经网络勾画较小靶区准确度[J].中国医学物理学杂志,2020,37(1):75.[doi:DOI:10.3969/j.issn.1005-202X.2020.01.015]
　JIANG Jialiang,LUO Yong,HE Yisong,et al.Feature area refocusing for improving the accuracy of small target area segmentations by fully convolutional networks[J].Chinese Journal of Medical Physics,2020,37(6):75.[doi:DOI:10.3969/j.issn.1005-202X.2020.01.015]

备注/Memo

备注/Memo:
【收稿日期】2022-12-20 【基金项目】国家自然科学基金（81830052）；上海市分子影像学重点实验室建设项目（18DZ2260400） 【作者简介】周信宏，硕士，研究方向：医学图像处理，E-mail: zhou_xh163@163.com 【通信作者】黄钢，博士，教授，研究方向：核医学分子影像，E-mail: huanggang@sumhs.edu.cn
更新日期/Last Update: 2023-06-28