Mixed scheme based multimodal medical image fusion ... - IEEE Xplore

1 downloads 0 Views 2MB Size Report
1 Email: {ikrajivsingh.gaur.richa}@[email protected]. 2Centre of Computer Education,. University of Allahabad, Allahabad, INDIA.
IEEE/OSA/IAPR International Conference on Infonnatics, Electronics & Vision

Mixed scheme based multimodal medical image fusion using Daubechies Complex Wavelet Transform Rajiv Singh!, Richa Srivastava!, Om Prakash!,2 and Ashish Khare! 1

Department of Electronics and Communication, University of Allahabad, Allahabad, INDIA 1 Email: {ikrajivsingh.gaur.richa}@[email protected] 2 Centre of Computer Education, University of Allahabad, Allahabad, INDIA. 2Email: [email protected] Abstract- Multimodal medical image fusion is an important task for

retrieving

complementary

information

from

different

modality of medical images. Image fusion can be performed using either spatial or transform domain methods.

Limitations of

spatial domain fusion methods led to transform domain methods. Discrete wavelet transform (DWT) based fusion is one of the most widely used transform domain method. But it suffers from shift sensitivity and does not provide any phase information. These disadvantages of DWT motivated us to use complex wavelet transform. In the present work, we have proposed a new multimodal

medical

image

fusion

method

using Daubechies

complex wavelet transform (DCxWT) which applies two separate fusion rules for approximation and detail coefficients.

Shift

invariance, availability of phase information and multiscale edge information properties of DCxWT improves the quality of fused image. We have compared the proposed method with spatial domain fusion methods (PCA and linear fusion) and transform domain fusion methods (discrete and lifting wavelet transforms). Comparison of results has been done qualitatively as well as by using

different

fusion

factor,

fusion fusion

metrics symmetry

(entropy, and

Q:rn

standard deviation, ).

On

the

basis

of

qualitative and quantitative analysis of the obtained results, the proposed method is found to be better than spatial domain fusion methods (PCA and linear fusion) and transform domain fusion methods (discrete and lifting wavelet transforms).

I.

INTRODUCTION

Medical image processing is a challenging and interesting area of research for scientists from last three decades with the invention of multimodal imaging sensors such as X-ray, CT(Computed Tomography), MRI(Magnetic Resonance Imaging) scanners etc. These inventions influenced the interest of researchers in medical imaging including a wide range of applications like denoising, registration, classification, segmentation and fusion. Unlike other applications, of medical imaging, image fusion [1] is an important application for extracting complementary information from different modality of medical images. Fusion of images is important for multimodal images, as single modal medical image provides only specific kind of information. For example, CT image provide the details of

978-1-4673-1154-0112/$3l.00 ©2012 IEEE

dense hard tissues and useful in tumor or anatomical detection, whereas MRI image gives information of soft tissues. Similarly Tl-MRI image provide details about anatomical structure of tissues whereas T2-MRI image gives information about normal and abnormal tissues. Thus we are not able to get all kind of information from one single modal medical image. Hence fusion of multimodal images is required. Generally image fusion [2] is defined as the process of combining multiple source images into single image which is more informative for human or machine perception. The basic requirement for image fusion is that it should carry all relevant information with reduced noise and it should not introduce any artifact during fusion process. Available literatures [3, 4] in this area proved the clinical importance of fusion of medical images. Image fusion methods can be grouped into three categories: Pixel level, Region level and Decision level. The present work uses pixel level fusion as it is simple and computationally efficient. There are two approaches to perform fusion process: spatial domain and transform domain. Spatial domain fusion techniques are simple and fused image can be obtained by directly applying fusion rules on pixel values of source images. Simple averaging, PCA (Principal Component Analysis) [5] and linear fusion [6] are some examples of spatial domain fusion techniques. But major disadvantage of spatial domain techniques are that they introduce spatial distortions in the fused image [7] and do not provide any spectral information. Transform domain techniques overcome the disadvantages of spatial domain fusion. Pyramid and wavelet transforms based techniques are the mostly used transform domain image fusion methods. Laplacian pyramid [8], contrast pyramid [9] and ratio of low pass pyramid [10] are popular methods of pyramid transform based image fusion. These methods overcome the disadvantages of spatial domain techniques but suffer from blocking effect [7]. As a result wavelet transform based fusion [II, 12] approaches are used and it was found that no blocking effect occurred during fusion process. Discrete wavelet transform (DWT) is widely used wavelet transform for image

ICIEV 2012

[EEE/OSA/[APR International Conference on [nfonnatics, E[ectronics & Vision fusion. It provides better directional selectivity in horizontal, vertical and diagonal directions and has better image representation. Maximum and weighted average fusion rules [7, 13] are applied for fusion of different source images using DWT. Another approach for image fusion is based on Lifting wavelet transform (LWT) [14] using modulus maxima criterion. The disadvantages of DWT or LWT are their shift sensitivity, poor directionality and lack of phase information [15]. DWT has been found shift sensitive [16] due to down sampling step in its implementation. Also DWT or LWT does not provide any phase information as it uses real filter banks. Due to these limitations of real valued DWT, complex wavelet transform based fusion techniques [17] are used. Complex wavelet transforms like Dual tree complex wavelet transform (DTCWT) [18] provides high directionality and shift invariance. But DTCWT has high computational requirement and due to redundancy it requires high memory. Also DTCWT is not a true complex wavelet transform because it uses real filter banks in its implementation. To overcome these limitations of DTCWT, an approximately shift invariant Daubechies Complex Wavelet Transform (DCxWT) [19, 20] is proposed and uses complex filter banks that makes it true complex wavelet transform. In the present work, we have proposed a new multimodal medical image fusion method using Daubechies complex wavelet transform (DCxWT) which applies two separate fusion rules for approximation and detail wavelet coefficients. Qualitative and quantitative comparison of the proposed method has been done with spatial domain fusion methods (PCA and Linear fusion) and wavelet transform (discrete and lifting wavelet transform) based fusion methods using different fusion performance measures. Rest of the paper is organized as follows: construction of Daubechies complex wavelet transform (DCxWT), in brief, is given in section II. Usefulness of DCxWT in image fusion is described in section III. Section IV explains the proposed fusion method. Experimental results and discussions are given in sections V. Finally, conclusions of the work are given in section VI. II.

DAUBECHIES COMPLEX WAVELET TRANSFORM (DCxWT)

Here If/(t) and

f/J(t)

share the same compact support

[-N,N+J], Any function fit) can be decomposed into complex scaling function and mother wavelet as: jmal-.

1

of/Ji ,k(t)+ I- dflf/j,k(t) o k FJo

f(t) = Lc£

(3)

where, jo is a given resolution level,

{c£G} and {df}

are

known as approximation and detailed coefficients. The Daubechies complex wavelet transform has the following advantages: (i). [t has perfect reconstruction, (ii). [t is non redundant wavelet transform, unlike Dual tree complex wavelet transform (DTCWT) [18] which has redundancy of 2m : I for m-dimensional signal. (iii), [t has the same number of computation steps as in DWT (although it involves complex computations), while DTCWT have 2m times computations as that of DWT for an m­ dimensional signals. (iv). It is symmetric. This property makes it easy to handle edge points during the signal reconstruction. III.

USEFULNESS OF DCxWT IN IMA GE FUSION

Daubechies complex wavelet transform exhibits following two important properties that directly improve the quality of fusion process.

A,

Reduced Shift Sensitivity

Daubechies complex wavelet transform is approximately shift invariant. A transform is shift sensitive if an input signal shift causes an unpredictable change in transform coefficients. In DWT shift sensitivity arises from use of downsamplers in the implementation. Figure 1 shows a circular edge structure reconstructed using real and complex Daubechies wavelets at single scale, From figure I, it is clear that as the circular edge structure moves through space, the reconstruction using real valued DWT coefficients changes erratically, while complex wavelet transfonn reconstructs all local shifts and orientations in the same manner. Shift invariance is desired during fusion process otherwise miss registration [21] problem will occur i.e. we will get mismatched or non aligned image.

The scaling equation of multiresolution theory is given by ¢(x) = 2 2: ak¢(2x-k) (I) k where ak are the coefficients. The ak can be real as well as complex valued and

L� =1 .

Daubechies's wavelet bases

{If/j,k(t)} in one dimension are defined through the above 2 scaling function and multiresolution analysis of L (RJ, To provide general solution, Daubechies considered ak to be real valued only. The construction details of Daubechies complex wavelet transform are given in [19], The generating wavelet If/(t) is given by n (2) If/(t)=2 L C-l) a1_,,¢C2t-n)

305

(c) (a) (b) Figure 1: (a) A circular edge structure, (b) reconstructed using wavelet coefficients of real-valued DWT at single scale and (c) reconstructed using wavelet coefficients of Daubechies complex wavelet transform at single scale,

ICIEV 2012

IEEE/OSA/IAPR International Conference on Infonnatics, Electronics & Vision B.

Availabiltiy of Phase Information

Daubechies complex wavelet transform provides phase information through its imaginary part of wavelet coefficients. The most of the structural information about images are contained in the phase of image. To show importance of phase we have taken two images, cameraman and medical image and decomposed those by DCxWT. Reconstruction of these images are done with exchanging the phase of these images with each other. From figure 2, it is clear that phase of image represents structural details or skeleton of the image. It was found that phase is an important criterion to detect strong features of images such as edges, comers etc. Hence by using Daubechies complex wavelet transforms, we are able to preserve more relevant information during fusion process and this will give better representation of fused image.

strong features (boundaries, corners) in a precise way and it was found experimentally that energy values of detailed wavelet coefficients carry the most of the structural information like edges and boundaries. Therefore, we have chosen energy based fusion method for detailed complex valued coefficients. In energy based fusion scheme, energy of each detailed band is computed and fused image is obtained by replacing detailed complex valued coefficients on the basis of energy values of detailed sub bands. The proposed fusion algorithm can be summarized as follows: (i). Decompose source images SICx,y) and S2Cx,y) using Daubechies complex wavelet transform (DCxW7) to obtain approximation AS/(x,y), AS2(x,y) and detail DS/(x,y), DS2(x,y) complex wavelet coefficients. Mathematically, [AS] (x,y) DS] (x,y)] = DCxWT[S] (x,y)] and

(4)

(ii). For approximation (i.e. ASj(x,y) and AS2(x,y», maximum fusion rule has been applied as below:

AS(x, Y )

=

A\\(X,Y) ifIAS� (X.}·)I>IAS;(X Y)1 { AS2 (x,y). ifIAS2 (x.y)I>IASj (x.y)1 •



.

.

.

(5)

where AS(x,y) is approximation level wavelet coefficient for fused image. (iii). For detail sub bands DS/x,y) (j is the total number of detail sub bands) of source images Slx,y) (i is the total number of source images), the energy of each sub band is denoted by EDSlx,y) and defined by:

[ DSj(x,y) ] I k=] n

EDSj(x,y) =

(6)

where k= 1,2, ... ,n is the maximum size of detail sub bands. If EDSj(x,y), EDS2Cx,y) are the energy of detail sub bands DSj(x,y), DS2Cx,y) for source images Sj(x,y) and Six,y) respectively, then selection of detail coefficients can be obtained by following rule:

(c) Figure 2: (a) cameraman image, (b) medical image, (c) image reconstructed from the phase of wavelet coefficients of cameraman image and modulus of wavelet coefficients of medical image and (d) image reconstructed from the phase of wavelet coefficients of medical image and modulus of wavelet coefficients of cameraman image.

IV.

DS(x,Y

) ={J),�[J,\,I(X'(X,Yy)').

(7)

where DS(x,y) is detail wavelet coefficient for fused image. (iv). Fused image F(x,y) is obtained by taking inverse Daubechies complex wavelet transform of AS(x,y) and DS(x,y), i.e. F(x,y) = Inverse DCxWT[AS(x,y) DS(x,y)] (8)

THE PROPOSED METHOD

We have proposed a new multimodal medical image fusion method by using Daubechies complex wavelet transform which uses two different fusion rules separately for scaling and detailed wavelet coefficients. Since scaling or approximation coefficients of an image represent the average information and a good fusion process should not lose this information, therefore we have chosen maximum fusion rule for scaling or approximation coefficients. Detailed complex valued coefficients carry information about strong features such as edges, corners etc. therefore we have to fuse the detailed coefficients in such a way that fused image will preserve edge information as well as represent

306

2

V.

RESULTS AND DISCUSSIONS

We have experimented with several medical images and here we have shown results on two different medical image data sets. The first set of medical image is MRI and CT image. Fusion result for first medical image data set is shown in figure 3. From results shown in figure 3 one can easily conclude that fused image obtained by the proposed method provides details of boney structures as well as information about soft tissues. Second set of medical images are TI-MR and MRA image showing some abnormality as calcified white structure in the image. TI-MR image provides the details of soft tissues but is unable to detect abnormality present in the

ICIEV 2012

IEEE/OSA/IAPR International Conference on Infonnatics, Electronics & Vision MRA image, similarly MRA image detects abnormality but unable to give information about soft tissues. From figure 4, it is clear that in this case the fused image obtained by the proposed method is more infonnative than the fused images obtained by the other fusion methods (PCA fusion [5], linear fusion [6], DWT based fusion [13] and LWT based fusion [14]). (a).TI-MR image

(b). MRA image

(c).The proposed fusion method (d).LWT based fusion [14]

(c).The proposed fusion method (d).LWT based fusion [14]

(e). DWT based fusion [13]

(t). Linear fusion [6]

(e). DWT based fusion [13]

(t). Linear fusion [6]

(g). PCA fusion [5] Figure 4: Fusion results for second set of medical images.

(g). PCA fusion [5] Figure 3: Fusion results for first set of medical images.

307

This visual representation is not sufficient for analysis of the obtained fusion results. Therefore we have compared the proposed method with other ones on quantitative measures. For comparison of the proposed method, we have used different fusion performance measures [5], [22]-[24] which are entropy, standard deviation, fusion factor, fusion symmetry and Q�,. Higher values of entropy, Q:,i' standard deviation, fusion factor indicate high quality fused image while lower value of fusion symmetry indicate good quality fused image. These fusion performance measures are tabulated in following two tables for two different medical imaging data sets:

ICIEV 2012

IEEE/OSA/IAPR International Conference on Infonnatics, Electronics & Vision Table I: Performance Comparison for first set of medical images Standard

Fusion

Fusion

Deviation

Factor

Symmetry

Q�n

6.0013

32.8511

3.3514

0.3669

0.6512

6.1921

34.0205

1.0883

0.2887

0.5809

Entropy

Method

The Proposed method LWT based method [14] DWT based method [13] Linearfusion [6] PCAfusion [5]

6.0640

33.3744

1.1189

0.3997

0.5764

5.6557

29.6860

2.2821

0.2553

0.6409

5.7638

28.3914

1.8034

0.4309

0.5278

uses mixed fusion scheme i.e. two separate fusion rules for approximation and details of complex wavelet coefficients. This fusion scheme is helpful for preserving average information of image as well as strong features such as edges, comers etc. Experiments were done with two different set of medical images. Performance evaluation of the obtained results are done with well defined fusion measures (entropy, standard deviation, fusion factor, fusion symmetry and Q�/i ). On the basis of quantitative and visual analysis, the proposed energy based fusion method is found to be better than other transform domain methods (LWT and DWT based methods) and spatial domain methods (PCA and linear fusion). Also the proposed method is able to preserve detail information (edges, boundaries) in better way. ACKNOWLEDGMENTS

Table 2: Performance Comparison for second set of medical images Entropy

Method

The Proposed method

5.9913

Standard

Fusion

Fusion

Deviation

Factor

Symmetry

Q�n

66.9296

4.3662

0.1576

0.5725

This work was supported in part by the Department of Science and Technology, New Delhi, India, under grant no. SRiFTP/ETA-023/2009 and the University Grants Commission, New Delhi, India, under grant no. 36246/2008(SR).

REFERENCES

LWT based method [14]

5.4987

43.6309

3.8142

0.1094

0.4564

5.3948

61.5091

3.0902

0.0155

0.4556

[I]

B.V. Darasthy, Information fusion in the realm of medical applications­ A bibliographic glimpse at its growing appeal, Information Fusion, voU3,pp. 1-9,2012.

DWT based method [13]

[2]

art,Guest editorial, Information Fusion,vol.8,pp. 114-118,2007. [3]

Linearfusion [6]

5.2756

38.3802

3.8227

0.0083

[5]

5.7417

56.6432

4.6493

0.2067

H. Schoder, H.W. Yeung, M. Gonen, D. Kraus, S.M. Larson, Head and Neck Cancer: Clinical Usefulness and Accuracy of PETfCT Image

0.3897

Fusion,Radiology,pp. 65-72,2004. [4]

PCAfusion

A. Goshtasby, S. Nikolov, Image Fusion: Advances in the state of the

Y. Nakamoto, K. Tarnai, T. Saga, T. Higashi, T. Hara, T. Suga, T. Koyama, K. Togashi,Clinical Value of Image Fusion from MR and PET

0.6402

in Patients with Head and Neck Cancer,

Molecular Imaging and

Biology, pp. 46-53, 2009. [5]

By observing data values of table 1 and table 2, it is clear that no method is better than other ones on each quantitative measure, but by considering the ranking of methods on each performance measure, it can be concluded that the proposed method excels majority of other methods In every perfonnance measure. Form the quantitative and visual analysis of the obtained results (i.e. comparison of fusion performance of figure 3 and figure 4 as well as measures in table 1 and table 2), it can be concluded that the proposed mixed fusion scheme based on Daubechies complex wavelet transform is better than other methods and is more beneficial for low contrast medical images. The proposed method preserves detail information (edge information) in better manner. VI.

CONCLUSIONS

Multimodal medical image fusion is an important task of medical imaging to retrieve complementary information from different modality of medical images. In the present work, we have proposed a new multimodal medical image fusion using Daubechies complex wavelet transform. The proposed method

308

V.P.S. Naidu, J.R. Rao, Pixel-level Image Fusion using Wavelets and Principal Component Analysis, Defence Science Journal, vol. 58, No. 3, pp. 338-352,2008.

[6]

J. G. P. W. Clevers, R. Zurita-Milla, Multisensor and multiresolution image fusion using the linear mixing model, Image Fusion: Algorithms and Applications,pp. 67-84,2008.

[7]

H. Li, B. S. Manjunath, S. K. Mitra,Multisensor image fusion using the wavelet transform,Graphical Models and Image Processing, vol. 57,no. 3,pp. 235-245,1995.

[8]

P. J. Burt, R. J. Kolczynski, Enhanced image capture through fusion, in Proceedings of the 4th IEEE International Conference on Computer Vision (ICCV '93),pp. 173-182,1993.

[9]

A. Toet, J. J. Van Ruyven, J. M. Valeton, Merging thermal and visual images by a contrast pyramid, Optical Engineering, vol. 28, no. 7, pp. 789-792,1989.

[10] A. Toet, Image fusion by

a

ratio

of

low-pass

pyramid, Pattern

Recognition Letters,vol. 9,no. 4,pp. 245-253,1989. [11] G. Pajares, 1. M. Cruz, A wavelet-based image fusion tutorial, Pattern Recognition,vol. 37,no. 9,pp. 1855-1872,2004.

[12] K.

Amolins, Y.

Zhang, P.

Dare, Wavelet

based

image fusion

techniques-an introduction, review and comparison, ISPRS Journal of Photogrammetry & Remote Sensing,vol. 62,no. 4,pp. 249-263,2007.

[13] C. Shangli, H.E. Junmin, L. Zhongwei, Medical Images of PETfCT Weighted Fusion Based on Wavelet Transform, Bioinformatics and Biomedical Engineering,pp. 2523-2525,2008. [14] S. Kor, U. S. Tiwary, Feature Level Fusion of Multimodal Medical Images in Lifting Wavelet Transform Domain,Engineering in Medicine

ICIEV 2012

IEEE/OSA/IAPR International Conference on Infonnatics, Electronics & Vision and Biology Society, EMBS '04. 26th Annual International Conference of the IEEE,vol. 1,pp. 1479 - 1482,2004. [15] A. Khare,

U.

S. Tiwary, M. Jeon, Daubechies complex wavelet

transform based multilevel shrinkage for deblurring of medical images in presence of noise, International Journal on Wavelets, Multiresolution and Information Processing,vol. 7,no. 5,pp. 587-604,2009. [16] A. Khare, M. Khare, Y. Y. Jeong, H. Kim, M. .leon, Despeckling of medical ultrasound images using complex wavelet transform based Bayesian shrinkage, Signal Processing, vol. 90, no. 2, pp. 428-439, 2010. [17] P. Hill, N. Canagarajah, D. Bull, Image fusion using complex wavelets, in Proceedings of the 13th British Machine Vision Conference, Cardiff, UK,2002. [18] 1. W. Selesnick, R. G. Baraniuk, N. G. Kingsbury, The Dual-Tree

Complex Wavelet Transform, IEEE Signal Processing Magazine, vol. 22,No. 6,pp. 123-151,2005.

[20] A. Khare,

U.

S. Tiwary, W. Pedrycz, M. Jeon, Multilevel adaptive

thresholding and shrinkage technique for denoising using Daubechies complex wavelet transform,The Imaging Science Journal,vol. 58,no. 6, pp. 340-358,2010. [21] Z.

Qiang, W.

Long, L.

Huijuan, M.

Zhaokun, Similarity-based

multimodality image fusion with shiftable complex directional pyramid, Pattern Recognition Letters,vol. 32,pp. 1544-1553,2011. [22] K. Kotwal,S. Chaudhuri,A novel approach to quantitative evaluation of hyperspectral image fusion techniques, Information Fusion (Article in Press),2011. doi:10.1016/j.inffus.2011.03.008 [23] S. Xydeas, V. Petrovic, Objective Image Fusion Performance Measure, Electronics Letters,vol. 36,no. 4,pp. 308-309,2000. [24] C. Ramesh, T. Ranjith, Fusion performance measures and a lifting wavelet transform based algorithm for image fusion, Proceedings of the Fifth International Conference on Information Fusion, vol. I, pp. 317-

320,2002.

[19] D. Clonda, 1. M. Lina, B. Goulard, Complex Daubechies Wavelets:

Properties and statistical image modeling,Signal Processing,vol. 84,pp. 1-23,2004.

309

ICIEV 2012