490

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 10, NO. 3, MAY 2013

Hybrid Pansharpening Algorithm for High

Spatial Resolution Satellite Imagery
to Improve Spatial Quality
Jaewan Choi, Junho Yeom, Anjin Chang, Younggi Byun, and Yongil Kim, Member, IEEE

AbstractMost pansharpened images from existing algorithms

are apt to present a tradeoff relationship between the spectral
preservation and the spatial enhancement. In this letter, we developed a hybrid pansharpening algorithm based on primary and
secondary high-frequency information injection to efficiently improve the spatial quality of the pansharpened image. The injected
high-frequency information in our algorithm is composed of two
types of data, i.e., the difference between panchromatic and intensity images, and the Laplacian filtered image of high-frequency
information. The extracted high frequencies are injected by the
multispectral image using the local adaptive fusion parameter and
postprocessing of the fusion parameter. In the experiments using
various satellite images, our results show better spatial quality
than those of other fusion algorithms while maintaining as much
spectral information as possible.
Index TermsHybrid pansharpening, primary and secondary
high-frequency information, spatial quality.

I. I NTRODUCTION

IGH spatial resolution satellite images, such as IKONOS,

QuickBird, KOMPSAT-2, Worldview-2, and Geoeye-1,
provide low spatial resolution multispectral images and high
spatial resolution panchromatic images. Generally, pansharpening methods aim to improve the spatial information of the
original multispectral image by using the spatial details of
the high spatial resolution panchromatic image in situations
where we cannot obtain ideal high spatial multispectral images
due to the technical limitations of certain satellite sensors [1].
Since Tu et al. [2] proposed the fast IHS fusion algorithm with
spectral adjustment, various pansharpening algorithms based
on the substitution and the addition of the high-frequency
information of panchromatic images have been developed.
Many users prefer substitution-based algorithms to addition
or multiresolution analysis (MRA)-based methods because the
former ones display better spatially enhanced images [3], [4].

The GramSchmidt (GS) sharpening, which was patented by

E. Kodak and implemented in the ENVI software, is one of the
most representative pansharpening methods [5]. Aiazzi et al. [6]
showed that the GS adaptive (GSA) method, which uses the
intensity image from multiple regression, is more efficient than
other GS-based fusion results and that the adaptive context
model of the GSA (GSA-CA), which is a local method of
the GSA, offers outstanding pansharpening quality. Combined
algorithms of generalized IHS (GIHS) and optimization methods have been also proposed [7][10]. These methods apply
an optimization formula for generating the optimal intensity
image, determining the suitable fusion parameter or extracting
high-frequency information. However, most algorithms have
a tendency to present a tradeoff relationship between minimizing spectral distortion and enhancing spatial information.
Pansharpening algorithms that minimize the spectral distortion
have trouble improving the spatial sharpness, whereas the GIHS
algorithm, which has an efficient result spatially, distorts the
spectral information.
The aim of this letter is to improve the spatial quality of
a pansharpened image while the spectral quality of pansharpening result is maintained close to that of existing pansharpening results. The general framework is briefly described in
Section II, and our algorithm is defined. In Section III, we
compare the quality of pansharpened images obtained by our
method with existing methods. Conclusions are presented in
Section IV.
II. H YBRID PANSHARPENING F RAMEWORK
A. General Pansharpening Framework
Aiazzi et al. [6] demonstrated a general fusion framework
that can be defined by
MShn = MSln + n (P h P l )

Manuscript received February 21, 2012; revised June 17, 2012; accepted
July 15, 2012. Date of publication September 6, 2012; date of current version November 24, 2012. This work was supported by the Korea Aerospace
Research Institute.
J. Choi is with the School of Civil Engineering, Chungbuk National University, Cheongju 361-763, Korea (e-mail: jaewanchoi@chungbuk.ac.kr).
J. Yeom, A. Chang, and Y. Kim are with the Department of Civil and
Environmental Engineering, Seoul National University, Seoul 151-742, Korea.
Y. Byun is with the Satellite Information Research Center, Korea Aerospace
Research Institute, Daejeon 305806, Korea.
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/LGRS.2012.2210857

(1)

where MShn is the pansharpened image of the nth band, MSln

is the resized multispectral image of the nth band, P h is the
panchromatic image, P l is the synthetic intensity image, which
has an equivalent spatial resolution with MSln , and n is the
global/local fusion parameter. Generally, a synthetic intensity
image obtained by combining the multispectral bands, such as
weighted averaging, increases the spatial quality of the fused
image better than a spatially degraded panchromatic image or
the addition of high-frequency information. In addition, n

1545-598X/$31.00 2012 IEEE

CHOI et al.: HYBRID PANSHARPENING ALGORITHM

491

influences the spectral/spatial quality of the fused image and

functions as a tradeoff parameter between spectral and spatial
distortions. Therefore, an appropriate determination of P l and
n is essential for generating the optimal pansharpened image.
B. Hybrid Pansharpening Algorithm
In this letter, we develop a new hybrid pansharpening
algorithm based on (1). The overall concept is given as
MShn = MSln + n [n Hn + n Hn ]

(2)

where n is the initial fusion parameter of the nth band, Hn and

Hn are the primary and secondary high-frequency information,
and n and n are the adjustment coefficients corresponding
to Hn and Hn , respectively. The high-frequency information in
(2) can be defined by
Hn = Pnh Inl


Hn = Lap Pnh Inl

(3)
(4)

where Pnh is the histogram-matched panchromatic image with

MSln , Lap() is the Laplacian filtering, and Inl is the adjusted
intensity image. The adjusted intensity image Inl is generated by
multiple regression between MSln and Pnh . The high-frequency
information obtained by (3) is similar to that of componentsubstitution-based pansharpening methods such as GIHS and
GS. Therefore, if n is set as zero, (2) is identical to (1). The
secondary high-frequency information in (4) plays a role that is
similar to the high boost filtering of general image processing
for improving the spatial edge of the image. It maximizes the
spatial clarity of the pansharpened image while maintaining
the spectral distortion caused by injecting the excessive highfrequency information. Fig. 1 shows an example of primary
and secondary high-frequency information corresponding to
the QuickBird imagery. The primary high-frequency information [see Fig. 1(b)] from (3) produces a fused image with
spatial characteristics similar to a panchromatic image. However, disparity could be injected into the homogeneous area of
each multispectral image as the number of the primary high
frequencies is increased due to the spectral residual between
the panchromatic and adjusted intensity images. Therefore, the
spectral distortion will be simultaneously increased, although
the increase in the amount of the primary high frequencies
improves the spatial quality of the fused image. On the contrary,
the secondary high frequencies in Fig. 1(c) accentuate the
dominant edge information of the feature compared with that in
Fig. 1(b), facilitating the enhancement of the additional spatial
resolution. The secondary high frequencies in Fig. 1(c) indicate
the second differential between panchromatic and intensity
images. In particular, the secondary high frequencies express
the supplementary edge of the panchromatic image, which is
represented as an additional differential form. Therefore, the
secondary high frequencies can improve the spatial quality of
the initial image through raising the pixel value of the image
edge or linear features in the viewpoint of spatial quality. In the
perspective of spectral distortion, it can maintain the tendency
of spectral information in the pansharpened image because the

Fig. 1. High-frequency information of hybrid pansharpening. (a) Panchromatic image, (b) primary high-frequency information of the blue band, and
(c) secondary high-frequency information of the blue band.

global structural properties of secondary high frequencies are

similar to those of primary high frequencies such as those in
Fig. 1(b) and (c). If the initial pansharpened image produced by
the primary high-frequency information shows good spectral
quality, our method can maintain the spectral quality of the
initial image.
C. Adjustment of the Local Fusion Parameter
To apply the hybrid pansharpening framework of (2) efficiently, the optimal fusion parameter must be determined.
Equation (2) requires three coefficients, i.e., n , n , and n .
We constructed these parameters empirically to minimize the
spectral distortion of the pansharpened image [11]. The initial
local gain coefficient n(x,y) is defined as
 


 MSln(x,y)

 Sn(x,y)
(5)
n(x,y) = Sn  
l
In(x,y)
where Sn and Sn(x,y) are the global and local correlation
values between the spatial information of a multispectral and
adjusted intensity image obtained by Laplacian filtering, and
() is the standard deviation. The value of the local gain
n(x,y) at (x, y) is computed by an 11 11 moving window.
However, a looped curve-shaped edge occurs in n(x,y) due to
numerical instabilities, such as shown in Fig. 2(a). To remove
this aspect, we postprocessed the local parameter. First, we
applied the Laplacian of a Gaussian (LoG) filter to n(x,y)
and then extracted looped curve-shaped edges that had a top
10% pixel value in the LoG-filtered image. The selective mean
filter (SMF) using a moving window, which has a size equal

492

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 10, NO. 3, MAY 2013

TABLE I
DATA S PECIFICATIONS

Fig. 2. Postprocessing of local parameters. (a) Initial local parameters that

have looped curve-shaped edges and (b) corrected parameters resulting from
the SMF and the median filter.

to that of n(x,y) , was applied to the extracted edges. This

filter smoothens the pixel values of the looped curve-shaped
edges and not the entire image. Finally, a 3 3 median filter
was applied to remove the adjacent outlier pixel values of the
smoothed image by the SMF. From this postprocessing, we can
correct the local distortion of n(x,y) while preserving almost
the entire n(x,y) value. Coefficients n and n are presented as



MSln
(I l )
 ,
, 
n = max
(I l ) MSln
N


if

n=1

1
N entropy

entropy(I l )
N

l

MSln

(I )
 , if

MSln

n=1


thr

1
N entropy

MSln

entropy(I l )

n = 0.5 (Hn )/ (Hn )


> thr

(6)
(7)

where entropy() indicates the image entropy of the zeroth

order corresponding to the image, N is the total band number
of the satellite image, and thr is the threshold value. Coefficient
n is designed as the global gain parameter for each sensor
by trial and error in order to minimize the spectral distortion
during high-frequency injection. Through experiments with
the various satellite sensors such as IKONOS, QuickBird,
and Geoeye-1, we assumed that the threshold value of 1.5
is the most efficient for pansharpening [11]. Meanwhile,
coefficient n restricts the number of secondary high-frequency
information. The amount of secondary high frequencies is
bigger than that of primary high frequencies because these
are described as an additional differential form. Using (7), the
deviation of image pixels into the secondary high frequencies
is modified according to the primary high frequencies. In the
experiments, we determined that half the number of the primary
high frequencies is the best optimal value for supplementing
insufficient spatial information of the initial image.
III. E XPERIMENTAL R ESULTS
In this letter, we used satellite images from QuickBird,
KOMPSAT-2, and Geoeye-1 of B, G, R, and near-infrared

multispectral and panchromatic bands to verify our algorithm.

The specifications of the data are described in Table I.
Our hybrid pansharpening method aims at improving the
visual sharpness while maintaining spectral information similar to other pansharpening algorithms. Meanwhile, component
substitution (CS)-based algorithms yield pansharpened images
of better spatial quality than MRA-based pansharpening and
other state-of-the-art methods [3], [4], [6], [12]. Therefore, we
compare our algorithm with representative CS-based pansharpening methods, such as the GIHS, GSA, GSA-CA, and adaptive
IHS (AIHS) algorithms. In the experimental comparison, determining how to quantitatively estimate the quality of pansharpened images is an important factor along with comparing the
pansharpening algorithms. General comparison methods using
spatially degraded data are not optimal for the original satellite
data because the experimental characteristics of the fused image
generated from spatially degraded data might be not congruous
with the characteristics of the original data. Therefore, we
adopted the D value of Khan et al. [12] to estimate the
quality of the pansharpened image. This index estimates the
similarity between the original multispectral image and the spatially degraded pansharpened image by using the Q4 index of
Alparone et al. [13] on the spectral quality estimation. In addition, we calculated the spectral index, such as relative dimensionless global error in synthesis (ERGAS), relative average
spectral error (RASE), and Qavg [8]. These indexes are computed using the original multispectral image and the spatially
degraded pansharpened image, which is similar to the process
of D . From the viewpoint of spatial quality estimation, the
spatial distortion index of Khan et al. [12] might be not efficient
enough to quantify the spatial sharpness of the fused image, as
it is evident in the visual comparison result of Khan et al. [12].
Therefore, we evaluated the spatial sharpness using the average
gradient (AG) among the pixel values of the pansharpened
image, along with the visual comparison, because it reflects the
contrasts of the detailed information of the pansharpened image
[14]. Table II shows the result of the evaluation indexes of the
pansharpened images. EXP indicates the resampled original
multispectral image. The lower the ERGAS, RASE, and D
and the higher AG and Qavg , the better the spectral and spatial
qualities of the pansharpened image. As shown in Table II, our
algorithm yielded a better spectral and spatial qualities than did
the GIHS, GSA, GSA-CA, and AIHS methods in the QuickBird
and Geoeye-1 images except for the ERGAS and the RASE of
the QuickBird data set. In particular, the tradeoff relationships
between D and AG or between Qavg and AG occurred in the
experimental data sets but not in the ERGAS and the RASE.

CHOI et al.: HYBRID PANSHARPENING ALGORITHM

493

TABLE II
C OMPARATIVE Q UICK B IRD /KOMPSAT-2/G EOEYE -1 PANSHARPENING R ESULT

Fig. 3. Detail of 300 300 of QuickBird. (a) Multispectral image and the
(b) GIHS, (c) GSA, (d) GSA-CA, (e) AIHS, and (f) hybrid results.

Fig. 4. Detail of 300 300 of KOMPSAT-2. (a) Multispectral image and the
(b) GIHS, (c) GSA, (d) GSA-CA, (e) AIHS, and (f) hybrid results.

That means that D and Qavg are more efficient indexes than
the ERGAS/RASE to estimate spectral quality. In the case
of the KOMPSAT-2 data set, the AIHS result had the best
spectral quality index value among the algorithms. In addition,

AIHS methods had the least AG value, and the fused image
by the AIHS of KOMPSAT-2 did not virtually include spatial
information of panchromatic imagery such as that in Fig. 4(e).
Therefore, our hybrid pansharpening result has better spatial

494

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 10, NO. 3, MAY 2013

IV. C ONCLUSION
In this letter, a hybrid pansharpening algorithm for improving
the spatial quality has been developed. By injecting highfrequency information of two types and postprocessing the
fusion parameter, we have empirically optimized the spectral
and spatial qualities of pansharpened images. From the experiments, the images obtained with our hybrid pansharpening had
better spatial sharpness and clarity than with the GIHS and
AIHS methods while retaining spectral characteristics closer
to the original multispectral image than those of the GS-based
methods. Through evaluation indexes and visual comparison,
the capability of our algorithm has been confirmed.
R EFERENCES

Fig. 5. Detail of 300 300 of Geoeye-1. (a) Multispectral image and the
(b) GIHS, (c) GSA, (d) GSA-CA, (e) AIHS, and (f) hybrid results.

sharpness than the other methods while overcoming the tradeoff problem between the spectral preservation and the spatial
sharpness of CS-based algorithms. As shown in Figs. 35, our
hybrid result has better spatial sharpness than the other methods
while maintaining color information visually similar to the
original multispectral image [see Figs. 3(f) and 4(f)]. In the case
of other fusion results, some color distortion occurred, such as
that in Figs. 3(b), 4(b), and 5(b). In addition, some blurring
occurred in some of the features, such as building edges and
moving cars [see Figs. 3(c)(e), 4(c)(e), and 5(c)(e)]. These
results show that hybrid pansharpening can improve the spatial
quality dramatically while preserving the spectral information
of the original multispectral image.

[1] Y. Zhang, Understanding image fusion, Photogramm. Eng. Remote

Sens., vol. 70, no. 6, pp. 653660, Jun. 2004.
[2] T. M. Tu, P. S. Huang, C. L. Hung, and C. P. Chang, A fast
intensityhuesaturation fusion technique with spectral adjustment for
IKONOS imagery, IEEE Geosci. Remote Sens. Lett., vol. 1, no. 4,
pp. 309312, Oct. 2004.
[3] B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, MTFtailored multiscale fusion of high-resolution MS and Pan imagery,
Photogramm. Eng. Remote Sens., vol. 72, no. 5, pp. 591596, May 2006.
[4] S. Baronti, B. Aiazzi, M. Selva, A. Garzelli, and L. Alparone, A theoretical analysis of the effects of aliasing and misregistration on pansharpened
imagery, IEEE J. Sel. Topics Signal Process., vol. 5, no. 3, pp. 446453,
Jun. 2011.
[5] C. A. Laben and B. V. Brower, Process for enhancing the spatial
resolution of multispectral imagery using pan-sharpening, U.S. Patent
6 011 875, Jan. 4, 2000.
[6] B. Aiazzi, S. Baronti, F. Lotti, and M. Selva, A comparison between
global and context-adaptive pan-sharpening of multispectral images,
IEEE Geosci. Remote Sens. Lett., vol. 6, no. 2, pp. 302306, Apr. 2009.
[7] A. Garzelli, F. Nencini, and L. Capobianco, Optimal MMSE pan sharpening of very high resolution multispectral images, IEEE Trans. Geosci.
Remote Sens., vol. 46, no. 1, pp. 228236, Jan. 2008.
[8] S. Rahmani, M. Strait, D. Merkurjev, M. Moeller, and T. Wittman, An
adaptive IHS pan-sharpening method, IEEE Geosci. Remote Sens. Lett.,
vol. 7, no. 4, pp. 746750, Oct. 2010.
[9] W. Yao and M. Han, Improved GIHSA for image fusion based on parameter optimization, Int. J. Remote Sens., vol. 31, no. 10, pp. 27172728,
Mar. 2010.
[10] D. Zhang and X. Zhang, Pansharpening through proportional detail injection based on generalized relative spectral response, IEEE Geosci.
Remote Sens. Lett., vol. 8, no. 5, pp. 978982, Sep. 2011.
[11] J. Choi, D. Han, and Y. Kim, Context-adaptive pansharpening algorithm
for high-resolution satellite imagery, Can. J. Remote Sens., vol. 38, no. 1,
pp. 109124, Feb. 2012.
[12] M. M. Khan, L. Alparone, and J. Chanussot, Pansharpening quality assessment using the modulation transfer functions of instruments,
IEEE Trans. Geosci. Remote Sens., vol. 47, no. 11, pp. 38803891,
Nov. 2009.
[13] L. Alparone, S. Baronti, A. Garzelli, and F. Nencini, A global quality measurement of pan-sharpened multispectral imagery, IEEE Geosci.
Remote Sens. Lett., vol. 1, no. 4, pp. 313317, Oct. 2004.
[14] S. Yang, M. Wang, and L. Jiao, Fusion of multispectral and panchromatic
images based on support value transform and adaptive principal component analysis, Inf. Fusion, vol. 13, no. 3, pp. 177184, Jul. 2012.