[1]陈健,张子豪,王甘红,等.基于不同深度学习架构建立结肠镜质量控制的人工智能辅助系统[J].中国医学物理学杂志,2024,41(11):1443-1452.[doi:DOI:10.3969/j.issn.1005-202X.2024.11.018]
　CHEN Jian,ZHANG Zihao,WANG Ganhong,et al.Constructing an artificial intelligence assisted system for colonoscopy quality control based on various deep learning architectures[J].Chinese Journal of Medical Physics,2024,41(11):1443-1452.[doi:DOI:10.3969/j.issn.1005-202X.2024.11.018]
点击复制

基于不同深度学习架构建立结肠镜质量控制的人工智能辅助系统()
分享到：

《中国医学物理学杂志》[ISSN:1005-202X/CN:44-1351/R]

卷:
41卷
期数:
2024年第11期
页码:
1443-1452
栏目:
医学人工智能
出版日期:
2024-11-26

文章信息/Info

Title:
Constructing an artificial intelligence assisted system for colonoscopy quality control based on various deep learning architectures
文章编号:
1005-202X（2024）11-1443-10
作者:
陈健1; 张子豪2; 王甘红3; 王珍妮1; 夏开建4; 徐晓丹1
1.常熟市第一人民医院/苏州大学附属常熟医院消化内科， 江苏 苏州 215500； 2.上海豪兄教育科技有限公司， 上海 200434； 3.常熟市中医院消化内科， 江苏 苏州 215500； 4.常熟市医学人工智能与大数据重点实验室， 江苏 苏州 215500
Author(s):
CHEN Jian1;  ZHANG Zihao2;  WANG Ganhong3;  WANG Zhenni1;  XIA Kaijian4;  XU Xiaodan1
1. Department of Gastroenterology, Changshu No.1 Peoples Hospital/Changshu Hospital Affiliated to Soochow University, Suzhou 215500, China 2. Shanghai Haoxiong Education Technology Co., Ltd., Shanghai 200434, China 3. Department of Gastroenterology, Changshu Hospital of Traditional Chinese Medicine, Suzhou 215500, China 4. Changshu Key Laboratory of Medical Artificial Intelligence and Big Data, Suzhou 215500, China
关键词:
深度学习; Transformer; 结肠镜质量控制; 结肠镜; 模型部署
Keywords:
Keywords: deep learning Transformer colonoscopy quality control colonoscopy model deployment
分类号:
R574.6
DOI:
DOI:10.3969/j.issn.1005-202X.2024.11.018
文献标志码:
A
摘要:
目的：利用不同深度学习架构模型构建结肠镜质量控制的深度学习模型，并深入探索其决策机制。方法：基于HyperKvasir和苏州大学附属常熟医院的数据集，筛选结肠镜图像，涵盖不同清洁度的肠道、息肉及盲肠。图像经过预处理和增强后，采用基于卷积神经网络（CNN）和Transformer的预训练模型进行迁移学习。模型训练采用交叉熵损失函数，使用Adam优化器，并实施学习率调度。为提高模型透明度，进行深入的可解释性分析，包括梯度加权分类激活映射、指导式梯度加权分类激活映射和沙普利加性解释等技术。最后，模型被转换为开放神经网络交换格式（ONNX）并部署到多种设备终端，以实现结肠镜质量的实时控制。结果：在3 831张结肠内窥镜图像中，EfficientNet模型在测试集上表现最佳，准确率达到0.992，超过其他CNN（DenseNet121、ResNet50、VGG19）和Transformer（ViT、Swin、CvT）架构模型，其精确率、召回率和F1值分别为0.991、0.989和0.990。在358张外部测试集图像上，EfficientNet模型的平均AUC、精确率和召回率分别为0.996、0.948和0.952。尽管模型整体表现出色，但仍存在一些误判情况。模型可解释性分析揭示其决策中所依赖的图像区域。此外，模型已成功转换为ONNX格式并在多种平台和设备上部署，实现每秒超过60帧的平均推理速度，确保结肠镜检查的实时质量控制。结论：本研究为结肠镜质量控制开发7种基于CNN与Transformer的模型，EfficientNet在各类别中展现出卓越性能，并已在多终端实现实时预测，为患者提供更高水平的医疗服务。
Abstract:
Abstract: Objective To develop deep learning models for colonoscopy quality control using various deep learning architectures, and to delve into the decision-making mechanisms. Methods The colonoscopy images were selected from two datasets separately constructed by the HyperKvasir and Changshu Hospital Affiliated to Soochow University, encompassing intestines of varying degrees of cleanliness, polyps, and cecums. After image preprocessing and enhancement, transfer learning was carried out using the pre-trained models based on convolutional neural network (CNN) and Transformer. The model training adopted cross-entropy loss functions and Adam optimizer, and simultaneously implemented learning rate scheduling. To enhance model transparency, a thorough interpretability analysis was conducted using Grad-CAM, Guided Grad-CAM, and SHAP. The final model was converted to ONNX format and deployed on various equipment terminals to achieve real-time colonoscopy quality control. Results In a dataset of 3 831 colonoscopy images, EfficientNet model outperformed the other models on the test set, achieving an accuracy of 0.992 which was higher than those of the other models based on CNN (DenseNet121, ResNet50, VGG19) and Transformer (ViT, Swin, CvT), with a precision, recall rate, and F1 score of 0.991, 0.989, and 0.990. On an external test set of 358 images, EfficientNet model had an average AUC, precision, and recall rate of 0.996, 0.948, and 0.952, respectively. Although EfficientNet model is high-performing, some misjudgments still occurred. Interpretability analysis highlighted key image areas affecting decision-making. In addition, EfficientNet model was successfully converted to ONNX format and deployed on multiple platforms and devices, and it ensured real-time colonoscopy quality control with an inference speed of over 60 frames per second. Conclusion Among the 7 models developed for colonoscopy quality control based on CNN and Transformer, EfficientNet demonstrated exemplary performance across all categories and is deployed for real-time predictions on multiple terminals, aiming to provide patients with better medical care.

相似文献/References:

[1]陶源,王佳飞,杜俊龙,等.基于卷积神经网络的细胞识别[J].中国医学物理学杂志,2017,34(1):53.[doi:10.3969/j.issn.1005-202X.2017.01.011]
　[J].Chinese Journal of Medical Physics,2017,34(11):53.[doi:10.3969/j.issn.1005-202X.2017.01.011]
[2]门阔,戴建荣. 利用深度反卷积神经网络自动勾画放疗危及器官[J].中国医学物理学杂志,2018,35(3):256.[doi:DOI:10.3969/j.issn.1005-202X.2018.03.002]
　MEN Kuo,DAI Jianrong. Automatic segmentation of organs at risk in radiotherapy using deep deconvolutional neural network[J].Chinese Journal of Medical Physics,2018,35(11):256.[doi:DOI:10.3969/j.issn.1005-202X.2018.03.002]
[3]邓金城,彭应林,刘常春,等. 深度卷积神经网络在放射治疗计划图像分割中的应用[J].中国医学物理学杂志,2018,35(6):621.[doi:DOI:10.3969/j.issn.1005-202X.2018.06.001]
　DENG Jincheng,PENG Yinglin,LIU Changchun,et al. Application of deep convolution neural network in radiotherapy planning image segmentation[J].Chinese Journal of Medical Physics,2018,35(11):621.[doi:DOI:10.3969/j.issn.1005-202X.2018.06.001]
[4]查雪帆,杨丰,吴俣南,等. 结合迁移学习与深度卷积网络的心电分类研究[J].中国医学物理学杂志,2018,35(11):1307.[doi:DOI:10.3969/j.issn.1005-202X.2018.11.013]
　ZHA Xuefan,YANG Feng,WU Yunan,et al. ECG classification based on transfer learning and deep convolution neural network[J].Chinese Journal of Medical Physics,2018,35(11):1307.[doi:DOI:10.3969/j.issn.1005-202X.2018.11.013]
[5]宫进昌,赵尚义,王远军. 基于深度学习的医学图像分割研究进展[J].中国医学物理学杂志,2019,36(4):420.[doi:DOI:10.3969/j.issn.1005-202X.2019.04.010]
　GONG Jinchang,ZHAO Shangyi,WANG Yuanjun.Research progress on deep learning-based medical image segmentation[J].Chinese Journal of Medical Physics,2019,36(11):420.[doi:DOI:10.3969/j.issn.1005-202X.2019.04.010]
[6]安莹,黄能军,杨荣,等. 基于深度学习的心血管疾病风险预测模型[J].中国医学物理学杂志,2019,36(9):1103.[doi:DOI:10.3969/j.issn.1005-202X.2019.09.021]
　AN Ying,HUANG Nengjun,YANG Rong,et al. Deep learning-based model for risk prediction of cardiovascular diseases[J].Chinese Journal of Medical Physics,2019,36(11):1103.[doi:DOI:10.3969/j.issn.1005-202X.2019.09.021]
[7]徐航,随力,张靖雯,等.卷积神经网络在医学图像分割中的研究进展[J].中国医学物理学杂志,2019,36(11):1302.[doi:DOI:10.3969/j.issn.1005-202X.2019.11.011]
　XU Hang,SUI Li,ZHANG Jingwen,et al.Progress on convolutional neural network in medical image segmentation[J].Chinese Journal of Medical Physics,2019,36(11):1302.[doi:DOI:10.3969/j.issn.1005-202X.2019.11.011]
[8]张富利,崔德琪,王秋生,等.基于深度学习和图谱库方法自动勾画肿瘤放疗中危及器官的比较[J].中国医学物理学杂志,2019,36(12):1486.[doi:DOI:10.3969/j.issn.1005-202X.2019.12.024]
　ZHANG Fuli,CUI Deqi,WANG Qiusheng,et al.Comparative study of deep learning- versus Atlas-based auto-segmentation of organs-at-risk in tumor radiotherapy[J].Chinese Journal of Medical Physics,2019,36(11):1486.[doi:DOI:10.3969/j.issn.1005-202X.2019.12.024]
[9]温佳圆,林国钰,张逸文,等.应用深度学习网络实现肾小球滤过膜超微病理图像的语义分割[J].中国医学物理学杂志,2020,37(2):195.[doi:DOI:10.3969/j.issn.1005-202X.2020.02.012]
　WEN Jiayuan,LIN Guoyu,ZHANG Yiwen,et al.Semantic segmentation of ultrastructural pathological images of glomerular filtration membrane using deep learning network[J].Chinese Journal of Medical Physics,2020,37(11):195.[doi:DOI:10.3969/j.issn.1005-202X.2020.02.012]
[10]秦楠楠,薛旭东,吴爱林,等.基于U-net卷积神经网络的宫颈癌临床靶区和危及器官自动勾画的研究[J].中国医学物理学杂志,2020,37(4):524.[doi:DOI:10.3969/j.issn.1005-202X.2020.04.023]
　QIN Nannan,XUE Xudong,WU Ailin,et al.Automatic segmentation of clinical target volumes and organs-at-risk in radiotherapy for cervical cancer using U-net convolutional neural network[J].Chinese Journal of Medical Physics,2020,37(11):524.[doi:DOI:10.3969/j.issn.1005-202X.2020.04.023]
[11]王安琪,于超,陈胤玮,等.基于LSTM-Transformer的脑电情感分析[J].中国医学物理学杂志,2024,41(12):1550.[doi:DOI:10.3969/j.issn.1005-202X.2024.12.013]
　WANG Anqi,YU Chao,CHEN Yinwei,et al.EEG emotion analysis based on LSTM-Transformer[J].Chinese Journal of Medical Physics,2024,41(11):1550.[doi:DOI:10.3969/j.issn.1005-202X.2024.12.013]

备注/Memo

备注/Memo:
【收稿日期】2024-06-19 【基金项目】苏州市科技发展计划（临床试验机构能力提升）项目（SLT2023006）；常熟市医学人工智能与大数据重点实验室能力提升项目（CYZ202301）；常熟市科技发展计划项目（CS202019, CSWS202316）；常熟市科技计划（社会发展）项目（CS202452） 【作者简介】陈健，副主任医师，研究方向：消化内镜人工智能，E-mail: szcsdocter@gmail.com 【通信作者】徐晓丹，主任医师，研究方向：医学人工智能、机器学习等，E-mail: xxddocter@gmail.com
更新日期/Last Update: 2024-11-26