Journal of Food Engineering 110 (2012) 345355

Contents lists available at SciVerse ScienceDirect

Journal of Food Engineering

journal homepage: www.elsevier.com/locate/jfoodeng

Detection of early bruises in apples using hyperspectral data and thermal imaging
Piotr Baranowski , Wojciech Mazurek, Joanna Wozniak, Urszula Majewska
Institute of Agrophysics, Polish Academy of Sciences, Doswiadczalna 4, 20-290 Lublin, Poland

a r t i c l e

i n f o

Article history:
Received 6 October 2011
Received in revised form 20 December 2011
Accepted 31 December 2011
Available online 8 January 2012
Keywords:
Apples and bruise
Hyperspectral imaging
Thermal imaging

a b s t r a c t
The early detection of bruises in apples was studied using a system that included hyperspectral cameras
equipped with sensors working in the visible and near-infrared (4001000 nm), short wavelength infrared (10002500 nm) and thermal imaging camera in mid-wavelength infrared (35005000 nm) ranges.
The principal components analysis (PCA) and minimum noise fraction (MNF) analyses of the images that
were captured in particular ranges made it possible to distinguish between areas with defects in the tissue and the sound ones. The fast Fourier analysis of the image sequences after pulse heating of the fruit
surface provided additional information not only about the position of the area of damaged tissue but
also about its depth. The comparison of the results obtained with supervised classication methods,
including soft independent modelling of class analogy (SIMCA), linear discriminant analysis (LDA) and
support vector machines (SVM) conrmed that broad spectrum range (4005000 nm) of fruit surface
imaging can improve the detection of early bruises with varying depths.
2012 Elsevier Ltd. All rights reserved.

1. Introduction
Detection of mechanical defects in apples is very important for
quality inspection systems. In spite of the fact that bruising is the
reason for rejecting the highest number of fruit in sorting lines, the
existing automatic sorting systems still lack precision in detecting
bruises and the manual sorting method is still used (Leemans et al.,
2002; Taoukis and Richardson, 2007; Xing et al., 2007). Bruising is
dened as damage of fruit tissue as a result of external forces
which cause physical changes of texture and/or chemical changes
of colour, smell and taste (Mohsenin, 1986). Two basic effects of
apple bruising can be distinguished, i.e. browning and softening
of fruit tissue. The susceptibility of apple to mechanical damage
depends on many factors, including soil cultivation, nutrition and
weather conditions in the eld during fruit growth (Woolf and
Ferguson, 2000).
Machine vision, which is an advanced technology to see objects with an assistance of computers, has been used in many
applications in agriculture, including pre- and postharvest product
quality and safety detection, and sorting (Kim et al., 2002; Brosnan
and Sun, 2004; Xing et al., 2005; Sun, 2008, 2010; Wang et al.,
2011). In recent years visual sorting systems have been applied
for detecting mechanical defects. They perform a multispectral or
hyperspectral analysis of colour to identify and classify damaged
areas and to nd correlation between spectral characteristics and
physico-chemical properties of healthy and affected tissues. Such
Corresponding author. Tel./fax: +48 81 7445061.
E-mail addresses: pbaranow@demeter.ipan.lublin.pl, pbaranow@ipan.lublin.pl
(P. Baranowski).
0260-8774/$ - see front matter 2012 Elsevier Ltd. All rights reserved.
doi:10.1016/j.jfoodeng.2011.12.038

systems use advanced procedures for image processing and analysis, including principal component analysis, partial least squares,
neural networks, linear discriminant analysis, support vector machines (Lu et al., 1999; Lu, 2003; Peng and Lu, 2005, 2006; Grahn
and Geladi, 2007; Wang et al., 2011).
Soft independent modelling of class analogy (SIMCA) is based
on making a PCA model for each class in a dened training data
set, consisting of samples with a set of attributes and their class
membership (Wold and Sjostrom, 1977). The term soft refers
to the fact that the classier can identify samples as belonging to
multiple classes and not necessarily producing a classication of
samples into non-overlapping classes. In SIMCA the observations
are projected into each PC model and the residual distances calculated. An observation is assigned to the model class when its residual distance from the model is below the statistical limit for the
class. The observation may be found to belong to multiple classes
and a measure of goodness of the model can be found from the
number of cases where the observations are classied into multiple classes. The training stage implies that one has identied enough samples as members of each class to be able to build a
reliable model. It also requires enough variables are measured to
describe the samples accurately. The test stage uses signicance
tests to classify new samples, where the decisions are based on statistical tests performed on the object-to-model distances. Each
model should be checked for possible outliers and improved if possible (as one would do for any PCA model) linear discriminant analysis (LDA) is a method used in machine learning to nd a linear
combination of features which characterize or separate two or
more classes of objects or events (Yang, 2010). The resulting combination may be used as a linear classier, or more commonly, for

346

P. Baranowski et al. / Journal of Food Engineering 110 (2012) 345355

Nomenclature
Ibl
Iim
Iwh
R
t
T
BR
NB

Intensity of dark image

Reectance intensity of a pixel of the hyperspectral image
Reectance intensity of the 99% reectance standard
white panel
Relative reectance of the hyperspectral image
Time, s
Temperature, K
Bruised region of the fruit
Nonbruised region of the fruit

dimensionality reduction before later classication. This supervised classication method provides a linear transformation of ndimensional feature vectors (or samples) into an m-dimensional
space (m < n), so that samples belonging to the same class are close
together but samples from different classes are far apart from each
other. The classication model is developed from Bayes rule
assuming the probability distribution within all groups is known,
and that the prior probabilities for groups are given, and sum to
100% over all groups. It is based on the normal distribution
assumption and the assumption that the covariance matrices of
the two (or more) groups are identical.
SVM is a pattern recognition method that is used widely in data
mining applications, and provides a means of supervised classication, as do SIMCA and LDA. SVM was originally developed for the
linear classication of separable data, but is applicable to nonlinear
data with the use of kernel functions. SVM are used in machine
learning, optimization, statistics, bioinformatics, and other elds
that use pattern recognition. The Unscrambler software ver. 10.1,
CAMO Software AS. Trondheim, Norway uses the algorithm based
on code developed and released under an modied BSD license
by Chih-Chung Chang and Chih-Jen Lin of the National Taiwan University (Hsu et al., 2009). SVM is a classication method based on
statistical learning wherein a function that describes a hyperplane
for optimal separation of classes is determined. As the linear function is not always able to model such a separation, data are
mapped into a new feature space and a dual representation is used
with the data objects represented by their dot product. A kernel
function is used to map from the original space to the feature
space, and can be of many forms, thus providing the ability to handle nonlinear classication cases.
The hyperspectral imaging was used on selected apple cultivars
to develop a multispectral technique which could be used for
scabs, fungal and soil contamination and bruising (Mehl et al.,
2002). By using either PCA or the absorption intensities at specic
frequencies three spatial bands were selected capable of separating
normal from contaminated apples. The correct classication of the
apples was found to vary from 76% to 95%, depending on the cultivars analysed. In the PCA method, bands obtained are linear combinations of the original spectral bands but are uncorrelated. The
rst PC band contains the largest percentage of data variance. For
each following PC band the variance decreases, much of which is
due to the noise in the original spectral data. With the objective
of detecting bruising an automated system was developed (Lu,
2003) to help the fruit industry provide better fruit for bruising
detection and to identify and segregate new and old bruises from
the normal tissues of apples. A bruise detection algorithm used
in this system included removing the background, normalization
to reduce the variations of reectance caused by an illumination
and the principal component analysis to enhance bruising features
and reduce data dimensionality. According to the author spectral

SH
DP
LDA
MNF
NETD
PCA
PPT
SIMCA
SVM

Region on the fruit surface with the shallower bruise

Region on the fruit surface with the deeper bruise
Linear discriminant analysis
Minimum noise fraction
Noise equivalent temperature difference
Principal component analysis
Pulsed-phase thermography
Soft independent modelling of class analogy
Support vector machine

region between 1000 and 1340 nm was the most appropriate for
detecting apple bruising. It was also suggested that the minimum
noise fraction (MNF) could be used instead of PCA. The PC bands
obtained are linear combinations of the original spectral bands
but are uncorrelated. The rst PC band contains the largest percentage of data variance. For each following PC band the variance
decreases, much of which is due to the noise in the original spectral
data. It was shown that the contaminated and damaged portions of
the apples were more distinguishable from the normal portions
when using the asymmetric second difference method for analyzing hyperspectral images of apple surface in the range of 424
899 nm (Mehl et al., 2004). It was stated that by using that method
there were no differences in the observations of the various apple
cultivars using the appropriate data treatment procedure.
The main problem to overcome is that existing sorting systems
are not capable of effectively distinguishing those fruits in which
bruising has occurred a short time before inspection (Xing et al.,
2005). This is because the majority of these systems analyse narrow
spectral ranges, e.g. reected visible light (400750 nm) or scattered near infrared radiation (1000 nm3 lm). They concentrate
on detecting fruit browning, which in the case of fruit with early
bruising may occur indistinctly or not at all (Samim and Banks,
1993; Ferguson et al., 1999; Xing et al., 2005). The application of
the near infrared spectroscopy method (7002200 nm) has shown
limited effectiveness for bruise detection in multicoloured apples,
e.g. Jonagold or Braeburn, and for early bruises (Upchurch et al.,
1994; Wen and Tao, 2000; Kleynen et al., 2003; Xing et al., 2005,
2007). The majority of apple bruise detection methods show deciencies in the case of dark skin colour or small surface area bruises.
It has been revealed that inherent surface morphology and skin coloration can signicantly affect the performance of the hyperspectral
systems if they are not properly addressed (Chen and Kim, 2004).
Because of the shortcomings of existing methods for early apple
bruise detection, there has been a growing interest in incorporating
multi-range non-destructive sorting methods. A set of reectance
and uorescence image data was proposed for discriminating between a healthy apple and an apple with fungal contamination
and bruising (Kim et al., 2001, 2002). Previous studies using thermal
imaging in mid-wavelength (MWIR) and long-wave infrared (LWIR)
for identifying tissue defects in fruit (including apple bruise detection) indicated that this method offers new possibilities, provided
that the process of heat conduction in the fruit can be precisely identied and the mechanism of heat contrast creation between the
bruised part and sound areas on the fruit surface is understood (Hellebrand et al., 2000; Maldague, 2001; Fito et al., 2004; Baranowski
et al., 2008; Baranowski et al., 2009).
In this paper the results of a joint application of hyperspectral
imaging (4002500 nm) and thermal imaging (35005000 nm)
are presented and the effectiveness of such a system for early
bruise detection and depth evaluation is discussed.

P. Baranowski et al. / Journal of Food Engineering 110 (2012) 345355

Specic aims of the paper are:

- to compare the effectiveness of the minimum noise fraction
(MNF) and the principal component analysis (PCA) for
extracting maximum apple bruise contrast from hyperspectral image sequences and from pulsed-phase method of
active thermography (PPT);
- to distinguish between bruised and unaffected tissues as
well as deeper and shallower bruises in apples of ve studied cultivars by utilizing supervised classication models
from hyperspectral and thermal data.
2. Materials and methods
2.1. Materials
The fruit material used in the study came from the same orchard belonging to the STRYJNO-SAD Fruit Producers Association,
situated 15 km from Lublin, Poland. Champion, Gloster, Golden
Delicious, Idared and Topaz apples (Malus domestica Borkh),
were collected directly after hand harvesting in 2011 and stored
for 15 h at 21 C and at 80% relative humidity. The fruits were preliminary graded according to the size and only apples of the diameters 78 cm of each cultivar were studied.
2.2. Analytical measurements
To calculate fruit bulk density, each apple was weighed with an
electronic digital Mettler XS1003S balance, Mettler Inc., Switzerland, operating at a capacity of up to 1000 g with readability of
0.001 g and the fruit volume was obtained through the measurement of water volume which was displaced by the fruit. Additionally, the fruit rmness was determined using a Lloyd LRX Universal
Testing Machine, produced by Lloyd Instruments Ltd., Hampshire,
UK, equipped with a tip of 11.3 mm in diameter. Three measurements of rmness were made on each fruit, in the pedicle area,
the middle part of the apple and in the calyx area. The mean of
these three rmness readings were expressed in N.
Soluble solids concentrations (SSC) were determined using a
Atago pocket refractometer produced by Atago Co., Ltd. Tokyo, Japan, at ambient temperature of 20 C. For each apple, two measurements of SSC were made at opposite sides of the fruit. The
obtained values were expressed in brix.
2.3. Bruising procedure
In order to check the effectiveness of detecting bruises that
reached down to various depths, the following bruising procedure
was applied. From 96 apples of each specimen, half were left without bruising, the rest were divided into two groups (24 apples in
each one) to be bruised at different depths. A position on the equatorial line on the surface of each apple was chosen for bruising. A
plastic roller with a diameter of 10 mm and a thickness of 1 mm
was put in this position and a cylindrical weight of 0.2 kg was
dropped (the contact surface was the base of the cylinder) from
either a height of 400 mm (for the group with deeper bruises) or
from a height of 200 mm (for the group with shallower bruises).
Then the fruits were stored at room temperature for 1 h before
the hyperspectral and thermal imaging bruise assessments.
To measure the bruise depth each apple was cut through the
centre of the bruise along the maximum diameter of the damaged
area in the direction of the centre of the fruit. The depth parameter
was calculated according to the method described by Menesatti
and Paglia (2001) as the distance along the axis perpendicular to
the surface and center of the fruit. The bruise depths calculated
with this method was compared and veried with the method

347

which based on active thermography, described in detail in Baranowski et al. (2009).

2.4. Hyperspectral imaging system
The following imaging spectrographs were used in the study: a
visible and near infrared (VNIR) ImSpector V10E imaging spectrograph (4001000 nm) and a short wavelength infrared (SWIR)
N25E 2/300 imaging spectrometer (10002500 nm) manufactured
by SPECIM, Finland. They were mounted 40 cm above a belt conveyor which had the speed regulated for each camera (to perform
line scanning of the fruit). The illumination source consisted of four
500 W halogen lamps. The system enabled the apples to be
scanned line-by-line as they moved through the eld of view of
the optical system on the conveyor belt. A hyperspectral image
was recorded for each scan. The resolution of the images acquired
by the VNIR and SWIR cameras differed. The resolution of the VNIR
camera image was 1344 (spatial) by 1024 (spectral) pixels by
12 bits, which corresponds to a root mean square (rms) spot radius
of less than 40 lm and a spectral resolution of 6.8 nm (with the default slit). The image from the SWIR camera had a resolution of 320
(spatial) by 256 (spectral) pixels by 14 bits, which corresponds to
an rms spot radius of less than 15 lm and a spectral resolution
of 10 nm (30 lm slit). The acquisition time of one scan of the fruit
surface for VNIR and SWIR cameras was 5 s. After nishing scanning an entire fruit, a data cube was obtained by selecting a rectangle covering the fruit area and eliminating background pixels from
the images of all bands. This procedure enabled the preliminary
reduction of the amount of data that required further processing.
The raw data from the cameras were calibrated to obtain the
reectance R by using the following equation (Sun, 2010):

Iim  Ibl
Iwh  Ibl

where Iim is the intensity of an image; Iwh is the intensity of the

white reference Spectralon plate with reectance of 99% (R99), Labsphere Inc. North Sutton, NH, USA; and Ibl is the intensity of the dark
image (the shutter closed, light sources turned off and the lens covered with a black cap).
2.5. MWIR imaging system
An active thermography system was designed, consisting of a
thermographic camera, an excitation source, and a system for controlling the heat pulse time as well as registration parameters and
external conditions in the thermostatted laboratory (air temperature controlled in the range 1530 C with the accuracy of
0.5 C). The SC7600 thermographic camera (FLIR Systems, Inc.,
USA) was used, which is sensitive in the Mid-wavelength Infrared
range (MWIR) of 35 lm. Using a detector (InSb) with the format
640  512 enabled the recording at 100 Hz in full resolution. The
systems thermal sensitivity NETD was 20 mK at an object temperature of 25 C. The spatial resolution of the camera is 1 mrad. Connection with a PC computer was possible via a Gigabit-Ethernet
port. A lens with an angular eld of view of 11  8.8 was used.
The camera was connected to a modular real time infrared system
IR-NDT (AT-Automation Technology GmbH, Trittau, Germany)
which supports all measurement techniques (lock-in, pulse, transient, etc.). The measurement of radiation temperature of the apples was done under controlled external conditions. All series of
measurements were performed at 21 C air temperature and relative humidity of 60% in daylight. The distance between the camera
lens and the apple surface was 0.5 m. Four infrared lamps PHILIPS
PAR 38 (IR Red) (175 W each) were used as the excitation source.
The infrared lamps were situated at a distance of 0.4 m from the

P. Baranowski et al. / Journal of Food Engineering 110 (2012) 345355

2.6. Analyzing algorithms

Two methods were used to reduce the dimensionality of the
data sets, to segregate noise components from the images and
especially to produce uncorrelated output bands for which bruise
discrimination would be possible. The rst was principal component analysis (PCA), which consists of nding a new set of orthogonal axes that have their origin at the data mean and that are
rotated so the data variance is maximized (Green et al., 1988;
Boardman and Kruse, 1994). The number of PC bands can equal
the number of bands in the original image; however, only the rst
few PC bands contain the majority of uncorrelated relevant information. The second studied method was minimum noise fraction
(MNF) transformation, which in fact is another, more complex, linear transformation that consists of two separate PCA rotations. The
rst rotation uses the PCs of the noise covariance matrix to decorrelate and rescale the noise in the data. This stage results in data in
which the noise has unit variance and no band-to-band correlations. In the second stage a standard PC transformation of the
noise-whitened data is performed. Additionally, it is possible to divide the data space into a part associated with large eigenvalues
and coherent eigenimages, and a complementary part with nearunity eigenvalues and noise-dominated images.
To distinguish the areas of the bruising from PCA and MNF
images a script was written in ImageJ software (Rasband, 1997
2011). The implementation of Otsu thresholding algorithm was
used. It divides the histogram in two classes and the inter-class
variance is minimized. The main stages of the thresholding procedure are presented in Fig. 1. The original 32-bit PC3 score of a

Temperature,oC

apple surface at the corners of a square whose sides were 0.38 m

long (Baranowski et al., 2009).

37

700

35

600

33

500

31

400

29

300

27

200

25

100

Power, W

348

23
0

Time, s
Fig. 2. Example of the thermal response of apple surface regions representing
bruised and sound tissues to rectangular heat pulse.

Golden Delicious apple (Fig. 1a) was converted into 8-bit image
and Otsu thresholding method was applied giving thesholded
areas distinguished in Fig. 1b as red regions. The plugin in ImageJ
software for particle nucleus counting was applied to this image
which enabled to distinguish regions of sizes ranging from 50 to
5000 pixels. This range was suitable for creating in majority of
cases the masks of bruised regions. In Fig. 1c such a mask is added
to the thresholded image whereas in Fig. 1d this mask is put on original image.
To get relevant information about fruit bruising in the range of
35 lm, the pulsed-phase thermography (PPT) was used (Baranowski et al., 2009; Ibarra-Castanedo and Maldague, 2004). In this
method the heat response signal can be presented as a superposition of the number of waves, each having different frequency,

Fig. 1. Illustration of thresholding procedure stages for PC3 score of Golden Delicious apple.

P. Baranowski et al. / Journal of Food Engineering 110 (2012) 345355

amplitude and phase delay. The decomposition of the response signal for has been done using the CooleyTukey algorithm of the fast
Fourier transform. The real and imaginary parts of the Fourier
transform has been used to calculate the phase (Ibarra-Castanedo
and Maldague, 2004). An individual rectangular heat pulse of the
length of 1 s was used and the characteristic thermal response of
the object to this pulse was analysed. The frame rate of
50 images/s and the number of frames in each sequence equal to
500 were chosen in this method. In Fig. 2 the rectangular heat
pulse characteristics (in Watts) and Golden Delicious apple surface temperature response in bruise and non-bruise regions are
presented. To make the difference between thermal responses of
these regions more readable the rst four seconds of the process
are presented in this gure.

349

Assuming that the validation should reect the prediction error

of the samples they were weighted considering standard deviation
of the population. The weight of 1/SDev was applied to all the variables (The Unscrambler Tutorials, 2006). This procedure was applied in all LDA and SIMCA models.
Analysis of the thermal response (MWIR spectral range) was
carried out in the frequency domain. Some image processing procedures performed in this study (segmentation, principal component analysis) were carried out using MATLAB 7.9.0, Natick,
Massachusetts, USA. The unsupervised classication models were
created with the use of The Unscrambler software ver. 10.1, CAMO
Software AS. Trondheim, Norway.

3. Results and discussion

2.7. The phase analysis of thermograms enables important information
to be obtained about the process of heat penetration within the studied
objects
The classication models for bruised (BR)/non bruised (NB) and
shallowly bruised (SH)/deeply bruised (DP) fruit including separately VNIR, SWIR and MWIR data as well as combinations of
VNIRSWIR and VNIRSWIRMWIR as variables were created
using the soft independent modelling of class analogy (SIMCA),
the linear discriminant analysis (LDA) and the support vector machine (SVM) methods. Both training and testing data sets for this
analysis contained 240 samples each. For each sample the areas
containing bruised or nonbruised regions were selected. The average of reectivity from all the pixels in these regions was calculated for all the bands separately. Each training and testing
datasets contained the equal number of samples for each cultivar.
The samples within cultivars were chosen randomly. To avoid low
signal-noise ratio, only the wavelengths ranging from 410 to
999 nm for VNIR and from 1005 to 2445 for SWIR were used for
classication. The MWIR variables (41) contained heat response
data in the rst 2 s (with time interval of 0.02 s) after individual
rectangular heat pulse lasting 1 s. The VNIR variables represented
data from 46 spectral channels with approximately 13 nm increments per pixel whereas the SWIR variables represented data from
47 channels with approximately 31 nm increments per pixel. The
quantile normalization was performed on the data to standardize
them before classication procedure. The quantile normalization
of three distributions (concerning VNIR, SWIR and MWIR data)
without a reference distribution was applied. It consisted in sorting
the distributions then setting to the average (arithmetical mean) of
the distributions. So the highest value in all cases becomes the
mean of the highest values, the second highest value becomes
the mean of the second highest values, and so on.
In this study, for creation of the SIMCA models the maximum of
rst seven PCs were considered. For creation of PCA components
the singular value decomposition algorithm was used, which is
preferable for not much extended data sets. This algorithm produces higher accuracy results but is not suited for data sets with
a high number both samples and variables since the algorithm always computes all the components. The cross validation method
was used for creating PCA components. The random method with
20 segments was applied.
In this study the Mahalanobis method was used to model the
classes in the LDA models. This method was chosen because the
variability within the groups was not the same structure and linear
method was not applicable.
In this study nu-SVC classication type was used and the kernel
type with the radial basis function. The nu-value assumed was 0.5
and gamma value was 0.0185. The cross validation was used with
two segments. The constant weight = 1 was applied all variables.

Physical properties of the studied cultivars are presented in

Table 1. Within analysed samples, the rmness varied from 25.1
to 62.6 N and the highest mean value of 52.1 N was noted for
Idared. This cultivar was characterized by the lowest value of soluble solid content (12.4 brix). The cultivars Topaz and Idared had
the highest values of bulk density, at 0.1366 and 0.1237 g/cm3,
respectively. The method of fruit bruising enabled considerable differences to be created between shallower and deeper bruises within all the studied cultivars. Shallower bruises ranged from 1.9 to
3.5 mm whereas deeper bruises were between 2.9 and 4.4 mm. It
was intended to nd out if differences of these properties between
cultivars affected the hyperspectral and thermal imaging results.
PCA of the apples of ve studied varieties revealed its usefulness for extracting the most relevant information from multiple
band imagery gathered in a few component images. Exemplary
principal component score images of the same Golden Delicious
apple in the VNIR and SWIR regions are presented accordingly in
Figs. 3 and 4.
These were constructed by replacing the intensity value of the
pixels in images with the score value of each principal component.
These gures present the rst six principal components which are
the most important from the point of view of bruise detection. The
uneven illumination conditions inuenced the rst four PC scores
of VNIR hyperspectral images and the rst three SWIR PC scores.
This result suggests that even illumination could be crucial for
using the rst PC scores in both regions of the spectrum.
The analysis of suitability of particular PC and MNF components was performed by examining the ability of their image display to increase the thresholding ability to separate between the
damage section and the rest of the apple skin. The most suitable
PC scores for bruise detection were PC4, PC5 and PC6 for the VNIR
camera (4001000 nm) and PC3, PC4 and PC5 for the SWIR (1000
2500 nm) camera (Table 2). PC2 and PC3 of VNIR bands displayed
more information related to the skin colour, especially differentiation of the skin colours (spots, blush). The percent of success in Table 2 was calculated by segmenting (based on thresholding) all the
images belonging to particular PC of a given cultivar and evaluating
the number of properly selected defects as a ratio of all the images
for this PC.
Incorporating SWIR bands into the PCA enabled not only better
detection of the browning tissue of the bruise but also gave additional information about skin spotting (Fig. 4).
The use of the MNF transform made the information about the
bruise localisation and composition even more readable (Figs. 5
and 6). For the VNIR range of wavelengths the most suitable components for bruise detection were MNF4, MNF5 and MNF6,
whereas for the SWIR bands, the components which were the most
independent from uneven illumination effects and could be used
for bruise detection were MNF4 and MNF5 (Table 2). In the SWIR
range the noise caused by the low input signal of some bands

350

P. Baranowski et al. / Journal of Food Engineering 110 (2012) 345355

Table 1
Physical properties of studied fruit and bruise depth response.
Cultivar

Statistics

Idared N = 30

Golden Delicious N = 30

Gloster N = 30

Champion N = 30

Topaz N = 30

Whole data set N = 150

Mean
Max
Min
SD
Mean
Max
Min
SD
Mean
Max
Min
SD
Mean
Max
Min
SD
Mean
Max
Min
SD
Mean
Max
Min
SD

Firmness (N)
Load at 8 mm
depth (N)

Load at maximum
load (N)

52.1
62.6
43.2
6.0
33.5
45.2
25.1
6.9
42.8
50.2
33.5
5.1
36.3
43.0
30.1
4.7
43.4
54.1
38.0
4.7
41.6
62.6
25.1
8.3

52.5
63.5
44.1
5.9
34.2
45.3
25.3
6.8
43.8
50.2
35.8
4.4
36.8
43.0
30.8
4.5
44.8
54.1
39.4
4.5
42.4
63.5
25.3
8.2

Soluble solid
content (brix)

Fruit density
(g/cm3)

Shallower bruise
depth (mm)

Deeper bruise
depth (mm)

12.4
13.3
11.4
0.7
13.6
15.4
12.4
1.2
13.7
15.0
12.8
0.9
13.9
14.8
13.1
0.5
13.6
14.0
13.3
0.3
13.4
15.4
11.4
1.0

0.1237
0.1281
0.1188
0.0041
0.1165
0.1195
0.1140
0.0027
0.1170
0.1196
0.1138
0.0023
0.1214
0.1249
0.1174
0.0030
0.1366
0.1447
0.1128
0.0050
0.1313
0.1447
0.1128
0.0092

2.5
3.5
1.9
0.5
2.6
3.0
2.2
0.3
2.6
3.3
2.1
0.4
2.5
3.4
2.0
0.6
2.6
2.9
2.4
0.2
2.5
3.5
1.9
0.4

3.6
3.9
3.4
0.2
3.4
3.8
3.1
0.3
3.0
3.5
2.9
0.2
4.0
4.4
3.5
0.4
3.8
4.0
3.6
0.2
3.6
4.4
2.9
0.4

Fig. 3. Principal component analysis (PCA) scores images for the VNIR wavelength range of an exemplary Golden Delicious apple.

had an inuence on the MNF components higher than MNF5 (in

Fig. 6 it is seen for MNF6). These components were not suitable
for bruise detection.
By using active thermography (PPT and lock-in), the registered
sequences were analysed, as well as creating ampligrams and
phasegrams for various frequencies. Compared to earlier study
(Baranowski et al., 2009) in which the applicability of PPT method
for bruise detection was evaluated, in the present study additionally the lock-in method was used. It occurred that both PPT and
lock-in methods are useful for early bruise detection. In Fig. 7 the
results of this analysis are presented for the Topaz apple. In this

gure, the thermogram marked with (a) is the image of the apple
before heat pulse or wave extinction (cold image). The fruit bruise
is not visible on this thermogram, which indicates that passive
thermography cannot be used for bruise detection. Images in (b
and c) are the results of lock-in analysis. The lock-in ampligram
is strongly inuenced by reections from the illumination source.
Therefore, the phase analysis, which is free of this inuence is preferable for defect recognition.
The images in (d, e and f) are phasegrams for various phase
shifts of the same fruit obtained by pulsed-phase thermography.
They were obtained with the use of the discrete Fourier transform.

351

P. Baranowski et al. / Journal of Food Engineering 110 (2012) 345355

Fig. 4. Principal component analysis (PCA) scores images for the SWIR wavelength range of an exemplary Golden Delicious apple.

Table 2
Results of PCA and MNF analysis for detecting bruises in the studied cultivars with the use of a standard image thresholding procedure.
Cultivar

Number
of samples

Champion
Gloster
Golden Delicious
Idared
Topaz

30
28
27
30
27

PC score most suitable for image

thresholding
VNIR

SWIR

PC4
PC4
PC6
PC4
PC5

PC4
PC3
PC3
PC3
PC5

PC thresholding
results % of success

89
88
87
93
86

MNF scores most suitable for

image thresholding
VNIR

SWIR

MNF5
MNF5
MNF4
MNF5
MNF6

MNF4
MNF4
MNF5
MNF4
MNF4

MNF thresholding
results % of success

93
90
90
97
87

Fig. 5. Minimum noise fraction transform (MNF) scores images for the VNIR wavelength range of an exemplary Golden Delicious apple.

352

P. Baranowski et al. / Journal of Food Engineering 110 (2012) 345355

Fig. 6. Minimum noise fraction transform (MNF) scores images for the SWIR wavelength range of an exemplary Golden Delicious apple.

Fig. 7. Scores of active thermography (35 lm) of Topaz apple. Red spot indicates the center of the bruise. (For interpretation of the references to colour in this gure legend,
the reader is referred to the web version of this article.)

The phase analysis of thermogram sequences by this method is

especially useful for the detection of defects at various depths beneath the fruit skin. It enables not only the elimination of the distortion of radiation temperature distribution resulting from the
heating by halogen lamps not being homogenous, but also the
identication of defects at various depths for component frequencies of the heat response.
The average reectance spectra obtained for the apples of four
cultivars in VNIR and SWIR ranges are presented in Fig. 8. Considerable differences occurred between these regions for the majority
of wavelengths studied (4002500 nm). In this gure the typical
spectra of ROIs were selected representing sound tissue as well
as shallow and deep bruise 1 h after bruising. The characteristic
water absorption bands that 660, 970, 1200, 1470 and 1900 nm

appear as localized minima. The carotenoids and chlorophyll pigments are reected through the absorption valleys around 500
and 680 nm (El Masry et al., 2008). In general the samples containing higher moisture contents had lower reectivity across their
spectra (Bunnik, 1978). The reectance from bruised surface, 1 h
after bruising was considerably lower than that from the normal
tissue over the entire spectral region but especially from 700 to
1900 nm. The difference in reectance between regions with varied bruise depths (shallower and deeper bruises) is also observed
in some parts of VNIR and SWIR ranges but it is cultivar dependent.
To acknowledge preferable ranges for supervised classication
from reectance spectral characteristic in VNIR and SWIR ranges
and from emission characteristics in MWIR range, the appropriate
LDA, SVM and SIMCA models were created as described in Section

353

P. Baranowski et al. / Journal of Food Engineering 110 (2012) 345355

1.0

healthy tissue

0.9

shallower bruise

0.8

shallower bruise

deeper bruise

0.7

deeper bruise

Relative reflectance

Relative reflectance

healthy tissue

0.6
0.5
0.4
0.3
0.2
0.1
0.0
400 600 800 1000 1200 1400 1600 1800 2000 2200 2400
Wavelength, nm

Wavelength, nm
1.0

healthy tissue

0.9

0.8

shallower bruise

0.8

shallower bruise

0.7

deeper bruise

0.7

deeper bruise

Relative reflectance

Relative reflectance

1.0

0.9

0.6
0.5
0.4
0.3
0.2

healthy tissue

0.6
0.5
0.4
0.3
0.2
0.1

0.1
0.0

0.0

400 600 800 1000 1200 1400 1600 1800 2000 2200 2400
Wavelength, nm

400 600 800 1000 1200 1400 1600 1800 2000 2200 2400
Wavelength, nm

Fig. 8. VNIR and SWIR spectral characteristic curves extracted from ROI pixels of the hyperspectral images representing healthy (not bruised), shallowly bruised and deeply
bruised tissue of: (a) Champion, (b) Gloster, (c) Golden Delicious, (d) Topaz apples.

Table 3
Classication results of LDA and SVM models for distinguishing healthy (NB not bruised) and bruised (BR) tissues based on VNIR and SWIR reected and MWIR emitted radiation
for the testing set of fruit samples.
Model

LDA N = 240
SVM N = 240

VNIR

SWIR

MWIR

VNIRSWIR

VNIRSWIRMWIR

NB

BR

Total

NB

BR

Total

NB

BR

Total

NB

BR

Total

NB

BR

Total

79
69

100
99

90
84

73
78

96
83

85
80

54
93

71
84

63
88

93
84

93
86

93
85

93
93

98
91

95
92

Table 4
Classication results of LDA and SVM models for distinguishing shallow (SH) and deep (DP) bruised tissues based on VNIR and SWIR reected and MWIR emitted radiation for
testing set of fruit samples.
Model

LDA N = 120
SVM N = 120

VNIR

SWIR

MWIR

VNIRSWIR

VNIRSWIRMWIR

SH

DP

Total

SH

DP

Total

SH

DP

Total

SH

DP

Total

SH

DP

Total

90
80

38
50

64
65

82
88

44
70

63
79

24
76

92
82

58
79

68
72

58
58

63
72

100
92

30
62

65
77

Table 5
Classication results of SIMCA model for distinguishing healthy (NB not bruised) and bruised (BR) tissues based on VNIR and SWIR reected and MWIR emitted radiation for
testing set of fruit samples.
Result of SIMCA model testing

TP
FP
FN

VNIR

SWIR

MWIR

VNIRSWIR

VNIRSWIRMWIR

NB

BR

Total

NB

BR

Total

NB

BR

Total

NB

BR

Total

NB

BR

Total

56
44
0

49
49
2

53
47
1

65
28
8

50
47
3

58
37
5

56
42
3

44
51
5

50
46
4

61
33
6

65
28
7

63
31
6

63
34
3

70
26
4

67
30
3

TP (true positives) cases which were recognized by their rightful class model (classied correctly).
FP (false positives) cases classied as belonging to two classes.
FN (false negatives) cases classied as belonging to a wrong class.

354

P. Baranowski et al. / Journal of Food Engineering 110 (2012) 345355

Table 6
Classication results of SIMCA model for distinguishing shallow (SH) and deep (DP) bruised tissues based on VNIR and SWIR reected and MWIR emitted radiation for testing set
of fruit samples.
Result of SIMCA model testing

TP
FP
FN

VNIR

SWIR

MWIR

VNIRSWIR

VNIRSWIRMWIR

SH

DP

Total

SH

DP

Total

SH

DP

Total

SH

DP

Total

SH

DP

Total

39
10
1

29
19
2

68
29
3

38
60
2

75
23
2

57
42
2

58
38
3

43
55
2

51
47
1

70
28
2

42
55
3

56
42
0

57
43
0

70
28
2

63
36
1

TP (true positives) cases which were recognized by their rightful class model (classied correctly).
FP (false positives) cases classied as belonging to two classes.
FN (false negatives) cases classied as belonging to a wrong class.

2. Both training and testing data sets contained the samples of apples of the ve investigated cultivars. The Tables 3 and 5 show the
rate of success (in per cent) obtained on the validation set to distinguish between sound (NB) and bruised (BR) tissues for LDA/SVM
and SIMCA models, respectively. In SIMCA models true positive
(TP), false positive (FP) and false negative (FN) scores are obtained
at signicance level of 5%. The best total scores for LDA-95%, for
SVM-92% and SIMCA-67% were obtained for models which incorporated VNIR, SWIR and MWIR variables jointly. The analysis of
the total scores for individual ranges (VNIR, SWIR or MWIR) indicated the lowest success rate for MWIR in the LDA model (63%)
and the highest for VNIR in the LDA model (90%). From among
the three studied classication methods the success score was
the lowest for the SIMCA models (true positive scores ranged from
50% to 67%, and false positive scores ranged from 30% to 47%). The
rate of success (in per cent) obtained on the testing data set to distinguish between shallow (SH) and deep (DP) tissue bruises for
LDA/SVM and SIMCA models are presented in Tables 4 and 6. The
best total scores were obtained in the SVM models including individual SWIR and VNIR variables (79% of success for each) and in the
SVM model for the whole range of VNIRSWIRMWIR. For the SIMCA models of shallow (SH) and deep (DP) bruise classication the
highest success rate was noticed for the model based on the VNIR
data (68% of success) and the whole VNIRSWIRMWIR data. It
should be indicated that the low number of the true positive scores
in SIMCA models could be improved by increasing signicance level, e.g. to 10% but in this case the number of false negative scores
increases. The obtained results of the supervised classication
show better performance of combination of spectral intervals
(from VNIRSWIRMWIR ranges) on the classication of normal
versus bruised apples but the same combination does not seem
to signicantly affect the classication between deep and shallow
bruises.
4. Conclusions
To detect early bruises in apples, a system was successfully
used, which incorporated the hyperspectral imaging of reected
radiation in VNIR and SWIR ranges and infrared thermal imaging
of emitted radiation in MWIR range. The whole spectrum range
(4005000 nm) studied was useful for detecting bruises created
one hour before the experiment.
Hyperspectral image analysis of VNIR and SWIR wavebands was
effectively performed by application of PCA. Even better results
were obtained by the use of the MNF transformation whose components could be preferable for image segmentation purposes.
Thermal MWIR imaging (30005000 nm) is useful for bruise
recognition when an active approach (lock-in or pulsed-phase) is
applied.
The analysis of the total scores for individual ranges (VNIR,
SWIR or MWIR) indicated lower prediction values than in case
when these ranges were included jointly into models. The created
models of supervised classication based on VNIR, SWIR and

MWIR ranges indicate that best prediction efciency for distinguishing bruised and sound tissues as well as bruises of various
depths is obtained for models incorporating these three ranges together. This suggests that it would be reasonable to consider
including MWIR range into sorting systems.

References
Baranowski, P., Lipecki, J., Mazurek, W., Walczak, R.T., 2008. Detection of watercore
in Gloster apples using thermography. Postharvest Biol. Technol. 47, 358366.
Baranowski, P., Mazurek, W., Witkowska-Walczak, B., Sawinski, C., 2009. Detection
of early apple bruise with the use of pulsed-phase thermography. Postharvest
Biol. Technol. 53, 91100.
Boardman, J. W., and Kruse, F. A., 1994, Automated spectral analysis: a geological
example using AVIRIS data, north Grapevine Mountains, Nevada: in
Proceedings, ERIM Tenth Thematic Conference on Geologic Remote Sensing,
Environmental Research Institute of Michigan, Ann Arbor, MI, pp. I-407I-418.
Brosnan, T., Sun, D.W., 2004. Improving quality inspection of food products by
computer vision-a review. J. Food Eng. 61, 316.
Bunnik, N.J.J., 1978. Multispectal reectance radiation by agricultural crops in
relation with their morphological and optical properties. H. Veenman & Zonen
B.V, Wageningen.
Chen, Y.R., Kim, M.S., 2004. Visible/NIR imaging spectroscopy for assessing quality
and safety of agrofoods. In: Davies, A.M.C., Garrido, A. (Eds.), Proceedings of the
11th International Conference on Near-infrared Spectroscopy. NIR publications,
Chichester, pp. 6775.
El Masry, G., Wang, N., Vigneault, C., Qiao, J., ElSayed, A., 2008. Early detection of
apple bruises on different background colors using hyperspectral imaging. LWT
41, 337345.
Ferguson, I., Volz, R., Woolf, A., 1999. Preharvest factors affecting physiological
disorders of fruit. Postharvest Biol. Technol. 15, 255262.
Fito, P.J., Ortol, M.D., De los Reyes, R., Fito, P., De los Reyes, E., 2004. Control of citrus
surface drying by image analysis of infrared thermography. J. Food Eng. 61 (3),
287290.
Grahn, H.F., Geladi, P. (Eds.), 2007. Techniques and Applications of Hyperspectral
Image Analysis. John Wiley & Sons Ltd..
Green, A.A., Berman, M., Switzer, P., Craig, M.D., 1988. A transformation for ordering
multispectral data in terms of image quality with implications for noise
removal. IEEE Trans. Geosci. Remote Sens. 26 (1), 6574.
Hellebrand, H.J., Linke, M., Beuche, H., Herold, B. & Geyer, M. 2000. Horticultural
products evaluated by thermography. AgEng, Warwick, 2626.
Hsu, C.W., Chang, C.C., Lin, C.J., 2009. A Practical Guide to Support Vector
Classication. Available from: <http://www.csie.ntu.edu.tw/~cjlin>.
Ibarra-Castanedo, C., Maldague, X.P., 2004. Pulsed Phase Thermography Reviewed.
QIRT J. 1 (1), 4770.
Kim, M.S., Chen, Y.R., Mehl, P.M., 2001. Hyperspectral reectance and uorescence
imaging system for food quality and safety. Transactions of the ASABE 44, 721
729.
Kim, M.S., Lefcourt, A.M., Chao, K., Chen, Y.R., Kim, I., Chan, D.E., 2002. Multispectral
detection of fecal contamination on apples based on hyperspectral imagery:
Part I. Application of visible and near-infrared reectance imaging. Transactions
of the ASABE 45, 20272037.
Kleynen, O., Leemans, V., Destain, M.F., 2003. Selection of the most efcient
wavelength bands for Jonagold apple sorting. Postharvest Biol. Technol. 30,
221232.
Leemans, V., Magin, H., Destain, M.F., 2002. On-line fruit grading according to their
external quality using machine vision. Biosyst. Eng. 83, 397404.
Lu, R., Chen, Y.R. & Park, B. 1999. Hyperspectral imaging for detecting bruises in
apples. ASAE Annual International Meeting. Paper No. 99-3120. Toronto,
Canada.
Lu, R., 2003. Detection of bruises on apples using near-infrared hyperspectral
imaging. Transactions of the ASAE 46 (2), 523530.
Maldague, X.P., 2001. Theory and Practice of Infrared Technology for
Nondestructive Testing. John Wiley & Sons, New York.
Mehl, P.M., Chao, K., Kim, M., Chen, Y.R., 2002. Detection of defects on selected apple
cultivars using hyperspectral and multispectral image analysis. Applied
Engineering in Agriculture 18, 219226.

P. Baranowski et al. / Journal of Food Engineering 110 (2012) 345355

Mehl, P.M., Chen, Y.R., Kim, M.S., Chan, D.E., 2004. Development of hyperspectral
imaging technique for the detection of apple surface defects and
contaminations. J. Food Eng. 61, 6781.
Menesatti, P., Paglia, G., 2001. Development of a drop damage index of fruit
resistance to damage. J. Agric. Eng. Res. 80 (1), 5364.
Mohsenin, N.N., 1986. Physical properties of plant and animal materials. Gordon &
Breach Science Publishers, New York, NY, USA.
Peng, Y., Lu, R., 2005. Modelling multispectral scattering proles for prediction of
apple fruit rmness. Trans. ASAE 48, 235242.
Peng, Y., Lu, R., 2006. An LCTF-based multispectral imaging system for estimation of
apple fruit rmness. Part II. Selection of optimal wavelengths and development
of prediction models. Trans. ASAE 49, 269275.
Rasband, W.S., 1997. ImageJ. US National Institutes of Health, Bethesda, Maryland,
USA, http://imagej.nih.gov/ij/.
Samim, M., Banks, N.H., 1993. Color changes in bruised apple fruit tissue. N.Z. J. Crop
Hort. Sci. 21, 367372.
Sun, D.-W., 2008. Computer Vision Technology for Food Quality Evaluation.
Academic Press/Elsevier, San Diego, California, USA, pp. 583.
Sun, D.-W. (editor) 2010., Hyperspectral Imaging for Food Quality Analysis and
Control, Academic Press / Elsevier, San Diego, California, USA, 496 pp., ISBN:
978-0-12-374753-2 (2010).
Taoukis, P.S., Richardson, M., 2007. Principles of Intermediate-Moisture Foods and
Related Technology. In: Barbosa-Cnovas, G.V., Fontana, A.J., Jr., Schmidt, S.J.,
Labuza, T.P. (Eds.), Water Activity in Foods: Fundamentals and Applications.
Blackwell Publishing, Ames, IA, USA.

355

The Unscrambler Tutorials 2006, Version 9.5, Camo A/S, Trondheim.

Upchurch, B.L., Throop, J.A., Aneshansley, D.J., 1994. Inuence of time, bruise-type,
and severity on near-infrared reectance from apple surfaces for automatic
bruise detection. Trans. Am. Soc. Agric. Eng. 37, 15711575.
Wang, J., Nakano, K., Ohashi, S., Kubota, Y., Takizawa, K., Sasaki, Y., 2011. Detection
of external insect infestations in jujube fruit using hyperspectral reectance
imaging. Biosys. Eng. 108, 345351.
Wen, Z., Tao, Y., 2000. Dual-camera NIR/MIR imaging for stem-end/calyx
identication in apple defect sorting. Trans. ASAE 43, 449452.
Wold, S., Sjostrom, M., 1977. SIMCA: a method for analyzing chemical data in terms
of similarity and analogy. In: Kowalski, B.R. (Ed.), Chemometrics Theory and
Application, American Chemical Society Symposium Series 52. American
Chemical Society, Wash, D.C., pp. 243282.
Woolf, A.B., Ferguson, I.B., 2000. Postharvest responses to high fruit temperatures in
the eld. Postharvest Biol. Technol. 21, 720.
Xing, J., Bravo, C., Jancsok, P., Ramon, H., De Baerdemaeker, J., 2005. Bruise detection
on Golden Delicious apples by using hyperspectral imaging with multiple
wavebands. Biosys. Eng. 90 (1), 2736.
Xing, J., Jancso, P., De Baerdemaeker, J., 2007. Stem-end/calyx identication on
apples using contour analysis in multispectral images. Biosyst. Eng. 96, 231
237.
Yang, Z.R., 2010. Machine Learning Approaches to Bioinformatics (Science,
Engineering, and Biology Informatics vol.4). World Scientic Publishing Co.,
Pte., Ltd., London, UK.