A robust feature-based registration method of multimodal image using

phase congruency and coherent point drift

Renbo Xia*a,b, Jibin Zhaoa, Yunpeng Liua,b
Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016;
b
Key Laboratory of Opto-Electronic Information Processing, Chinese Academy of Sciences,
Shenyang 110016
a

ABSTRACT
This paper presents a new feature matching algorithm for nonrigid multimodal image registration. The proposed
algorithm first constructs phase congruency representations (PCR) of images to be registered. Then scale invariant
feature transform (SIFT) method is applied to capture significant feature points from PCR. Subsequently, the putative
matching is obtained by the nearest neighbour matching in the SIFT descriptor space. The SIFT descriptor is then
integrated into Coherent Point Drift (CPD) method so that the appropriate matching of two point sets is solved by
combining appearance with distance properties between putative match candidates. Finally, the transformation estimated
by matching the point sets is applied to registration of original images. The results show that the proposed algorithm
increases the correct rate of matching and is well suited for multi-modal image registration.
Keywords: Multimodal registration, feature matching, phase coherence, coherent point drift

1. INTRODUCTION
Due to the wide availability of different imaging modalities, multimodal registration is emerging as a promising research
area in recent years. Multimodal registration is a process of determining the alignment of two images from different
imaging modalities. Typically, one image, called the reference image, is considered the reference to which the other
image, called floating image, is compared. Multimodal registration is more challenging than monomodal registration
because multimodal images have different gray level characteristics and the features in the two images to be registered
are often not well preserved. So far, extensive researches have been made on image registration technique, and various
registration methods for medical image have been developed. A good survey of the existing literature on this problem
can be found in [1-3]. The image registration methods are generally classified into two mains categories: intensity-based
methods and feature-based methods.
Intensity-based registration methods don't require extracting features in images and attempt to find directly pixel
correspondences between two images by determining the similarity between two pixel points according to feature
information from neighboring pixel points. Some common similarity metrics used in intensity-based methods include
maximum likelihood [4], correlation [5, 6], and mutual information (MI) [7-9]. Of particular interest in recent years are
techniques based on mutual information, which attempt to match data points by finding the mutual dependence between
the images. Such techniques are very powerful and therefore are considered to be the state-of-the-art approaches to
multimodal image registration. An excellent survey of these methods, and their variations, is in [10]. The main advantage
of MI is that it allows for the direct intensity-based comparison of multimodal images, making no underlying
assumptions regarding the intensity relationships between the images under evaluation. However, since the intensity
relationship between the images is relatively unconstrained by MI, this cost function can be highly non-monotonic with
many local maxima. Since local optimization schemes are dependent on the monotonicity of the underlying cost function
such schemes can often get trapped in local maxima. This is particularly problematic in situations with small initial
overlaps, where the optimization scheme must travel a long distance to the correct solution. This need for careful
initialization of the system to achieve proper registration is one of the drawbacks to the use of MI. Furthermore,
techniques based on mutual information are computationally expensive and may not be practical for certain situations
where computational speed is important.
*xiarb@sia.cn; fax +86-24-23970054
MIPPR 2013: Pattern Recognition and Computer Vision, edited by Zhiguo Cao, Proc. of SPIE
Vol. 8919, 891903 2013 SPIE CCC code: 0277-786X/13/$18 doi: 10.1117/12.2031615

Proc. of SPIE Vol. 8919 891903-1

Downloaded From: http://reviews.spiedigitallibrary.org/ on 03/31/2014 Terms of Use: http://spiedl.org/terms

Feature-based methods extract a number of corresponding features (such as points, lines, contours, invariant moments,
orientation, and shape properties) between the pair of images to be registered [11-15]. Knowing the corresponding
features between two images, a transformation is then determined to map the floating image to the reference images,
thereby establishing pixel-by-pixel correspondence between the floating and reference images. Obviously, feature-based
methods are data dependent. Identification of the organs of interest in multimodal images is not always an easy task and
inaccuracies in feature extraction process have severe effects on a subsequent registration step, preventing partly featurebased methods from being used generally in multimodal image registration. On the other hand, frequently feature-based
methods are more accurate and less sensitive to photometric variations than intensity-based methods as long as the
feature extraction is reliable and accurate. Due to the reduction of the problem space, feature-based methods are also
significantly faster to compute, and are more suitable for real time applications. So, this paper will concentrate on the
registration approach based on feature matching.
Robust structure descriptors are invariant to image scaling, rotation, translation, variable lightning conditions, as well as
noise. One particular approach that satisfies these requirements is local phase congruency (PC). Morrone et al. [16]
firstly observed that important visual features are perceived at points of maximum local phase congruency, where the
phases of different spatial frequencies are aligned. PC has become a very appealing concept with some advantages: it has
bee demonstrated experimentally [17] that phase spectrum of images, rather than the amplitude spectrum, contains much
of the perceptually important information in an image; the phase is stable not only under translation but also geometric
deformations and contrast variations. Morrone et al. [16] defined PC at each location as the amplitude weighted mean
local phase angle of all the Fourier terms at that point and identified PC from the extremum points of local energy.
Kovesi [19, 20] extended the concept to two dimensional signals and proposed a more robust energy model which was
calculated in different orientations in order to address issues such as noise. PC gives good results for a wide range of
synthetic and natural images and produces feature maps that compare favourably with e.g. a Canny edge detector [18,
19]. Recently, there is an obvious trend to use PC model to image processing and a lot of researches on image
registration have been done by using the PCRs of images. Dalvi and Abugharbieh [21] proposed a registration approach
which first computes PC information from the slices/volume using oriented 2D Gabor wavelets and then extracts a set of
feature points that are subsequently matched using an Iterative Closest Point (ICP) approach. Wong et al. have published
a series of papers to show the wide availability of PC to multimodal image registration [22-25]. Particularly, Wong and
Fieguth [23] proposed a PC based feature extraction method to determine the locations of sub-clouds to be used as
keypoints for medical image registration.
The primary goal of this paper is to develop a robust and fast approach to multimodal medical image registration that
provides comparable accuracy at a significantly reduced computational cost. The proposed approach can be divided into
several parts. First, the PCRs of the images to be aligned is constructed and then the well known SIFT [26] method is
applied to capture significant feature points from PCRs, as described in section II. Second, an iterative scheme is
designed to solve robustly the simultaneous matching and feature space transformation estimation problem by combing
CPD algorithm, recently introduced in [27], with the SIFT descriptor in section III. Finally a set of experiments are
reported to demonstrate the capability of our method in Section IV. Concluding remarks are presented in Section V.

2. FEATURE EXTRACTION BASED ON PHASE CONGRUENCY AND SIFT

DESCRIPTOR
2.1 Phase congruency representations of the images
Images captured with different modalities can possess significantly different pixel intensity mappings for the same
content, making them difficult to compare in a direct manner. One possible solution to this problem is to construct
representations of the multimodal images that allow for comparison independent of the underlying image intensity. PC
has been proved to be an effective method for determining structural significance in an image [19, 20].
PC adopted in this paper is based on the method proposed by Kovesi can be described as follows. The local amplitude
and phase at each point in image I is computed over multiple scales and orientations using complex-valued wavelets
such as logarithmic Gabor wavelets. The local amplitude and phase at a particular point x at wavelet scale n is
computed based on a pair of even-symmetric and odd- symmetric wavelets H ne and H no at scale n
An (x) =

( I ( x) H ) + ( I ( x) H )
e 2
n

o 2
n

Proc. of SPIE Vol. 8919 891903-2

Downloaded From: http://reviews.spiedigitallibrary.org/ on 03/31/2014 Terms of Use: http://spiedl.org/terms

(1)

 I (x) H ne
o
I ( x) H n
The PC at point x , orientation , and over a range of N scales can then be computed as

n (x) = tan 1

PC (x, ) =

N
n

W (x, ) An (x, ) (x, ) Th

N
n

An (x, ) +

(x, ) = cos(n (x, ) (x, ))

sin(n (x, ) (x, ))

(2)

(3)

(4)

where W represents the frequency spread weighting factor, An and n represent the amplitude and phase at wavelet
scale n respectively, represents the weighted mean phase, T represents the noise threshold and is a small constant
used to avoid division by zero. In this paper, PC (x) is referred to as the phase congruency representation of image.
2.2 Feature extraction using SIFT descriptor
Feature extraction is a crucial step in image registration. A feature extractor should have good spatial localization and be
robust against noise and geometric/photometric changes. Although PC may be used directly to detect corner points,
feature points do not necessarily correspond to physical corner points. This is true particularly in multimodal medical
image registration.
Recently, methods to extract scale and affine invariant interest points have been proposed. A review of these methods
can be found in [28]. A comprehensive evaluation of the interest point detectors is provided in [29]. Among these
methods, the SIFT method is among the best performers and can achieve illumination invariance in addition to scale
invariance. However, the SIFT method is highly sensitive to data non-homogeneities, where the same data content is
represented by different data point values due to certain conditions. For example, in the case of MRI data volumes, RF
inhomogeneity conditions can result in significant data non-homogeneities in the constructed data volume. Fortunately,
the PC measure is largely invariant to data non-homogeneities. Therefore, The combination of PC with SIFT is a natural
choice because of their complementary characteristics.
In this section, a feature extraction method based on PC and SIFT, namely PC-SIFT, is implemented to improve the
robustness of feature extraction against inhomogeneity. The PCR of image first is constructed adopting the method
mentioned above. Then feature points are determined from this PCR of image using SIFT method. SIFT involves two
stages: feature extraction and description. The description stage concerns use of the low-level features in point matching,
and this will be considered next section.

3. SIMULTANEOUS FEATURE MATCHING AND SPACE TRANSFORMATION

ESTIMATION
3.1 Finding initial matching pairs using SIFT descriptor
Feature matching is used to determine the corresponding features in different datasets that represent homonymous
entities in the real world. Lowe [26] presented a simple initial matching scheme based on the descriptor vectors of the
feature points. Given the feature points detected in two images I and J , A feature point p n from image I is
considered a match to a feature point q m from image J if p n is the nearest neighbor of q m in the descriptors feature
space and
pn qm

p qm

'
n

where

<2

(5)

denotes the Euclidean distance between the descriptors of the two feature points and p 'n is the second nearest

feature point of p n in image J . For simplicity, we refer to this matching algorithm as the nearest neighbor algorithm. In
our experiments, a threshold = 0.8 is selected, which eliminates 90% of the false matches while discarding less than
5% of the correct matches.

Proc. of SPIE Vol. 8919 891903-3

Downloaded From: http://reviews.spiedigitallibrary.org/ on 03/31/2014 Terms of Use: http://spiedl.org/terms

3.2 Simultaneous feature matching and space transformation estimation based on CPD algorithm
The above matching scheme searches feature matches in the descriptors feature space which does not involve location
information of feature point, so many mismatches may occur. Therefore, it is necessary to develop a matching algorithm
based on location information of feature points. Many algorithms exist for point sets matching. A popular one is the
Iterative Closest Point (ICP) algorithm [30], which iterates two steps until it reaches the local minimum. However ICP
requires that the initial positions of the two point sets be adequately close, which is not always possible, especially when
transformation is non-rigid. Furthermore, ICPs performance degenerates quickly with outliers, even if some robustness
control is added [31]. Recently, Myronenko and Song [27] introduced a novel probabilistic method called as CPD for
point sets registration. This method treats the registration as a Maximum Likelihood (ML) estimation problem with
motion coherence constraint over the velocity field such that one point set moves coherently to align with the second set.
In this paper, CPD method is preferred because it outperforms than most state-of-the-art algorithms in the presence of
noise, outliers and distortion.
The CPD method takes two data sets as input representing feature points of a reference image I and a floating image J .
Let X N D = (x1 ,L , x N )T and YM D = (y1 ,L , y M )T be two input data sets containing N and M feature points
respectively. The goal is to compute the parameters of the transformation T (Y) that best aligns the transformed points
YT = T (Y ) , to the point set X . For nonrigid point sets registration, the transformation is defined as
T ( Y, v ) = Y + v ( Y )
(6)
where v is a displacement function. The objective function of registration model takes the form
N D
1 N M +1 old

2
2
(7)
Q (v , 2 ) =
P (m xn ) xn (y m + v(y m )) + p log 2 + Pv
2 2

n =1 m =1

where P old (m x n ) denotes posterior probabilities of Gaussian mixture model (GMM), N p = n =1 m =1 P (m x n ) N ,

operator P extracts a part of the function for regularization. The detailed explanation can be found in reference [27].
A drawback of the CPD method is that it isnt immune from the presence of false match correspondences or outliers.
To deal with this problem, in this paper extra information like SIFT descriptor is incorporated to the CPD method to
improve the robustness of feature matching stage. For distinction, we call the extended CPD method as SIFT-CPD
method.
Next, we show how to integrate SIFT and CPD methods to overcome one of the intrinsic limitation of each algorithm.
The matching stage of SIFT provides a putative initial set of correspondences which can be used to initialize SIFT-CPD
method. The objective function equation (7) of the original CPD method is modified to consider both appearance
similarity and geometrical proximity properties between feature points by
N D
1 N M +1 old

2
2
2
(8)
Q (v , 2 ) =
P (m xn ) ( (1 ) xn (y m + v(y m )) + pn qm ) + p log 2 + Pv
N

2 2

n =1 m =1

The coefficient is a factor that control the weight of SIFT based appearance in matching stage. We have observe that
a good tradeoff to weight appearance and distance information is obtained by setting the coefficient = 0.4 . The closed
form solution of (8) can be directly obtained by adopting the same scheme with the reference [27].
In general, the proposed SIFT-CPD method works very well on our test data and achieves impressively high
performance. However, it faces a few challenges due to the high complexity of test data. Firstly, the SIFT-CPD method
might fail to find the optimal correspondences due to the interference of mismatches when the initial matching based on
the SIFT descriptors is not satisfactory. Secondly, in most cases, more than a third of the feature points in our data has no
correspondences. To match them takes a lot of time. Therefore, we propose a new scheme to eliminate false match pairs.
If the displacement transformation between two point sets is given by SIFT-CPD method, we can project the feature
point y m in one set to the other one and find its nearest neighbor point x n . If the nearest neighbor x n is just y m , we
will say that a pair ( x n , (v(y m ) ) is a matching candidate. The matching measure between x n and y m is defined as

follows
2

cnm = (1 ) x n (y m + v(y m )) + p n q m

(9)

So, the proposed removal scheme can be summarized as follows

Step 1. Compute the nearest neighbors for all elements in v(Y) .
Step 2. Select all matching candidate pairs and delete all unselected points.
Step 3. Compute the matching measure for selected candidate pairs by using equation (9). The measure becomes the
cost of removing that pair.

Proc. of SPIE Vol. 8919 891903-4

Downloaded From: http://reviews.spiedigitallibrary.org/ on 03/31/2014 Terms of Use: http://spiedl.org/terms

Step 4. Place all the pairs in a heap keyed on cost with the maximum cost pair at the top.
Step 5. Remove the pairs whose cost is greater than given threshold . In our test, the is set as 1.5. A small value
limits matching candidates to a reduced neighborhood while a bigger value allows matching more distant features.
The above removal scheme is used after the SIFT-CPD method is carried out. In generally, the removal scheme and the
SIFT-CPD method can be used alternately in iterative process.
Finally, the estimated transformation by registering the point sets is applied to registration of original images.

4. RESULTS
In this section, we demonstrate the capabilities of the proposed PC-SIFT feature extraction method and SIFT-CPD
feature matching method for image registration. The simulation results have been obtained using the MATLAB software
package. We first look at the correct ratio of matching of our algorithms on feature points. We then examine the
effectiveness of the matching method to multimodal image registration.

Li

(a)

(b)

(c)

(e)

(d)

(f)

Fig 1. Example of feature extraction and matching: (a) floating image (T1-weighted MR), (b) reference image (T2-weighted MR), (c)
and (d), corresponding PCRs of floating image and reference image, (e) feature matching result obtained directly using the SIFT
method, (f) feature matching result obtained using the proposed SIFT-CPD method.

Fig. 1 illustrates the feature matching results of applying the proposed method and Lowes SIFT to the brain MRI T1-T2
images. Fig. 1(a) and (b) represent the original T1 and T2 images respectively. The PCRs of the two images are
demonstrated in Fig. 1(c) and (d). It is seen that the two PCRs are very similar. Fig. 1(e) and (f) show the putative
matching result found by SIFT descriptor and the result obtained by our algorithm. In this experiment the SIFT-CPD
algorithm clearly outperforms the original SIFT algorithms and gives reliable matching result.
In Fig. 2, we report a registration experiment with a pair of MRI images obtained using a T1 and T2 weighted imaging
protocols respectively. Each image was obtained from different subjects making it apt for the application of a non-rigid
deformation. For visual evaluation of the proposed method, the registered floating image and the fused images between
the reference and the transformed floating images are presented in Fig. 2(g) and (h) respectively. It can be seen from the
results that the proposed algorithm provides a robust matching of multimodal images while achieving good accuracy
visually.

Proc. of SPIE Vol. 8919 891903-5

Downloaded From: http://reviews.spiedigitallibrary.org/ on 03/31/2014 Terms of Use: http://spiedl.org/terms

(a)

(b)

(c)

(e)

(d)

(f)

(g)
(h)
Fig. 2. Example of multimodal images using the proposed mehtod: (a) floating image (T1-weighted MR), (b) reference image (T2weighted MR), (c) and (d), corresponding PCRs of floating image and reference image, (e) feature matching result obtained directly
using the SIFT method, (f) feature matching result obtained using the proposed SIFT-CPD method, (g) registered result applying the
estimated transformation to floating images, (h) fused image by (b) and (g).

5. CONCLUSION
In this paper, we presented a robust feature based nonrigid registration algorithm for multimodal image data. The new
algorithm first extracts feature points from PCRs of the input images by using SIFT method. Initial matches of feature
points are obtained by the nearest neighbor algorithm in descriptor space. Then the SIFT descriptor as appearance
similarity is integrated into the CPD algorithm to refine the initial matches. Finally, the estimated transformation by
matching the point sets is applied to registration of original multimodal medical images. The algorithm was tested on
several data and illustrated via two examples. Experimental results show that the proposed SIFT-CPD algorithm is robust
to data non-homogeneities and is well suited for multi-modal medical image registration.

ACKNOWLEDGMENT
This work was supported by the National Natural Science Foundation of China under Grant No. 51005229, and the

Proc. of SPIE Vol. 8919 891903-6

Downloaded From: http://reviews.spiedigitallibrary.org/ on 03/31/2014 Terms of Use: http://spiedl.org/terms

residential Foundation of the Chinese Academy of Sciences under Grant No. 07AR210201.

REFERENCES
[1] L.G. Brown, A survey of image registration techniques, ACM Comput. Surv, 24(4), 325-376 (1992).
[2] J.B. AntoineMaintz, M.A. Viergever, A survey of medical image registration, Med. Image Anal, 2 (1), 1-36
(1998).
[3] B. Zitova, J. Flusser, Image registration methods: a survey, Image Vis. Comput. 21(11), 977-1000 (2003).
[4] W. Li, H. Leung, A maximum likelihood approach for image registration using control point and intensity, IEEE
Transactions on Image Processing, 13(8), 1115-1127 (2004).
[5] A. Fitch, A. Kadyrov, W. Christmas, J. Kittler, Fast robust correlation, IEEE Transactions on Image Processing,
14(8), 1063-1073 (2005).
[6] Z. Li, Z. Bao, H. Li, G. Liao, Image autocoregistration and InSAR interferogram estimation using joint subspace
projection, IEEE Transactions on Image Processing, 44(2), 288-297 (2006).
[7] A. Collignon, F. Maes, D. Delaere, D. Vandermeulen, P. Suetens, G. Marchal, Automated multi-modality image
registration based on information theory, Proceedings of the Information Processing in Medical Imaging, 263-274
(1995).
[8] W.M. Wells, P. Viola, H. Atsumi, S. Nakajima, R. Kikinis, Multi-modal volume registration by maximization of
mutual information, Med. Image Anal, 1(1), 35-51 (1996).
[9] A. Studholme, D.L.G. Hill, D.L. Hawkes, An overlap invariant entropy measure of 3D medical image alignment,
Pattern Recogn, 32, 71-86 (1999).
[10] J.P.W. Pluim, A. Maintz, M.A. Viergever, Mutual information based registration of medical images: a survey,
IEEE Trans. Med. Imaging, 22(8), 986-1004 (2003).
[11] F. Eugenio, F. Marques, J. Marcello, A contour-based approach to automatic and accurate registration of
multitemporal and multi- sensor satellite imagery, Proceedings of the IEEE International Geoscience and Remote
Sensing Symposium, 6, 3390-3392 (2002).
[12] V. Govindu, C. Shekhar, Alignment using distributions of local geometric properties, IEEE Transactions on
Pattern Analysis and Machine Intelligence, 21(10), 1031-1043 (1999).
[13] A. Wong, W. Bishop, J. Orchard, Efficient multi-modal least-squares alignment of medical images using quasiorientation maps, in: Proceedings of the International Conference on Image Processing, Computer Vision, and
Pattern Recognition, June 2006.
[14] X.L. Dai, S. Khorram, A feature-based image registration algorithm using improved chain-code representation
combined with invariant moments, IEEE Transactions on Geoscience and Remote Sensing, 37(5-2), 2351-2362
(1999).
[15] M. Brown, D. Lowe, Automatic panoramic image stitching using invariant features, International Journal of
Computer Vision, 74 (1), 59-73 (2007).
[16] M. C. Morrone, R. A. Owens, Feature detection from local energy, Pattern Recognition Letters, 6, 303-313
(1987).
[17] A.V. Oppenheim, J.S. Lim, The importance of phase in signals, Proc. of IEEE, 69, 529541 (1981).
[18] F.J. Madrid-Cuevas, R. Medina-Carnicer, A. Carmona-Poyato, and N.L. Fernandez-Garcia, Dominant points
detection using local phase, Pattern Recognition and Image Analysis, 4478, 138-145 (2007).
[19] P. Kovesi, Image features from phase congruency, Journal of Computer Vision Research, 6, 2-26 (1999).
[20] P. Kovesi, Phase congruency a low level image invariant, Psychological Research, 64, 136-148 (2000).
[21] R. Dalvi, Abugharbieh R. Fast feature based multi slice to volume registration using phase congruency, Conf
Proc IEEE Eng Med Biol Soc, 5390-5393 (2008).
[22] A. Wong, David Clausi, ARRSI: Automatic Registration of Remote Sensing Images, in IEEE Transactions on
Geoscience and Remote Sensing, 45(5), 1483-1493 (2007).
[23] A. Wong, Paul Fieguth, Fast Phase-based Registration of Multimodal Image Data, in Signal Processing, 89, 724737 (2009).
[24] A. Wong, Jeff Orchard, Robust Multimodal Registration using Local Phase Coherence Representations, in
Journal of Signal Processing Systems: Special Issue on Biomedical Imaging (Springer), 54(1), 89-100 (2009).
[25] A. Wong, David Clausi, and Paul Fieguth, CPOL: Complex phase order likelihood as a similarity measure for
MR-CT registration, Medical Image Analysis,14(1), 50-57 (2010).

Proc. of SPIE Vol. 8919 891903-7

Downloaded From: http://reviews.spiedigitallibrary.org/ on 03/31/2014 Terms of Use: http://spiedl.org/terms

[26] D. Lowe, Distinctive image features from scale-invariant keypoints, International Journal of Computer Vision,
60(2), 91-110 (2004).
[27] A. Myronenko, Song X., "Point-Set Registration: Coherent Point Drift," Technical Report, ( 2009)
[28] K. Mikolajczyk, C. Schmid, Scale and affine invariant interest point detectors, International Journal of Computer
Vision, 60(1), 63-86 (2004).
[29] K. Mikolajczyk, C. Schmid, A performance evaluation of local descriptors, IEEE Transactions on Pattern
Analysis & Machine Intelligence, 27(10), 1615-1630 (2005).
[30] P.J. Besl, McKay N.D.. A method for registration of 3-d shapes, IEEE Trans. Pattern Anal. Mach. Intell., 14(2),
239-256 (1992).
[31] H. Chui, Rambo J., Duncan J., Schultz R., Rangarajan A., Registration of cortical anatomical structures via robust
3Dpoint matching, in: Information Processing in Medical Imaging (IPMI), Springer, Berlin, 168-181 (1999).

Proc. of SPIE Vol. 8919 891903-8

Downloaded From: http://reviews.spiedigitallibrary.org/ on 03/31/2014 Terms of Use: http://spiedl.org/terms