Abstract

Info

Title:: Advances in multimodal medical image fusion method based on deep learning

Author(s):: XUE Zhanqi; WANG Yuanjun; Institute of Medical Imaging Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China

Keywords:: medical image image fusion deep learning convolutional neural network deep belief network

Abstract:: The medical image fusion method can integrate useful information into a single image to increase the amount of information in a single image.When multimodal medical images are fused, how to effectively transform the images, extract the special features in different images, and apply appropriate fusion rules becomes the focus of research on medical image fusion. In recent years, with the rapid development of deep learning, deep learning has been widely used in medical images by replacing some manual operations in traditional methods and has shown unique advantages in image representation, image feature extraction and the selection of fusion rules. Herein the medical image fusion based on deep learning is discussed. Several deep learning methods that are commonly used in the framework of medical image fusion, such as convolutional neural network, convolutional sparse representation, deep self-encoding and deep belief network are introduced. Some deep learning methods that are applied to different steps of the fusion process are also summarized. Finally, the disadvantages of recent researches on the image fusion based on deep learning are analyzed, and the research direction in future is forecasted.