Abstract

Info

Author(s):: RUI Yingying; KONG Xiangyong; LIU Yanan; DONG Xin; CAI Jian; LU Yanzhuan; KUANG Zhongling; School of Medical Instrument and Food Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China

Keywords:: Keywords: Mask Scoring R-CNN deep learning transfer learning tooth-marked tongue tongue classification

Abstract:: Abstract: Objective To identify tongue features based on Mask Scoring R-CNN and transfer learning. Methods After the features were extracted by convolutional neural network, the backbone networks of ResNet-101 and feature pyramid network were used to extract features from low-level and high-level networks, and the levels of pyramid features were plotted according to different proportions. Region generation network was then used to generate candidate regions of interest from the features extracted from the backbone network. Finally, tooth marks in each region of interest were detected and segmented. Results The test on the test set with 232 samples showed that F1 score was 0.95, and that the accuracy rate, precision rate and recall rate of the proposed method were 0.93, 0.99 and 0.914, respectively. Conclusion Using the proposed method can effectively identify the features of tooth marks, accurately locate the position of tooth marks, calibrate the size of tooth marks, and extract the number of tooth marks on small-sample tongue image data set. The proposed method which has good effectiveness, generality and generality provides a basis for the severity analysis of tooth marks and serves as an objective and convenient computer-assisted tongue diagnosis method for monitoring disease progression from the perspective of disease prevention, mobile healthcare or bioinformatics.