Medical Image Fusion Based on Deep Convolutional Neural Network

Abstract

Background: Medical image fusion plays an important role in helping doctors for effective diagnosis and treatment. Objectives: The purpose of image fusion is to combine information from various different medical modalities into a single image with preserving salient features and details of the source image. Methods: In this article, we present an approach for fusion MRI and CT images based on a deep convolutional neural network with four layers that was trained with medical images. In the beginning, images were decomposed to high and low frequencies by applied nonsubsampled shearlet transform (NSST). Then, for high-frequency sub-band, we used deep convolution neural networks for extracting feature maps. Low-frequency sub-band became fusion using the law of local energy fusion and in the end, the fused images were reconstructed by reverse NSST. Results: Experimental results indicated that the proposed scheme had better functionality in terms of image preservation, visual quality, and subjective and objective assessment. Conclusion: In this work, a medical image fusion method based on deep convolutional neural networks was proposed. The main novelty of this approach was the use of a deep convolutional neural network with four layers that was trained to extract source image features. To achieve good results, we used the nonsubsampled shearlet transform technique for multi-scale decomposition. Based on the experimental results, the proposed method achieved the best fusion performance.