Pattern Recognition 37 (2004) 965976

www.elsevier.com/locate/patcog
Color texture classication by integrative
Co-occurrence matrices
Christoph Palm
1
Institute of Medical Informatics, Aachen University of Technology, D-52057 Aachen, Germany
Received 26 November 2002; received in revised form 4 August 2003; accepted 18 September 2003
Abstract
Integrative Co-occurrence matrices are introduced as novel features for color texture classication. The extended
Co-occurrence notation allows the comparison between integrative and parallel color texture concepts. The information prot
of the new matrices is shown quantitatively using the Kolmogorov distance and by extensive classication experiments on
two datasets. Applying them to the RGB and the LUV color space the combined color and intensity textures are studied
and the existence of intensity independent pure color patterns is demonstrated. The results are compared with two baselines:
gray-scale texture analysis and color histogram analysis. The novel features improve the classication results up to 20% and
32% for the rst and second baseline, respectively.
? 2003 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.
Keywords: Color texture; Co-occurrence matrix; Integrative features; Kolmogorov distance; Image classication
1. Introduction
Texture analysis is one of the main topics in the eld of
digital image processing. Texture is described as a pattern
with some kind of regularity. The approaches of mathemat-
ical modeling are grouped into structural, statistical and sig-
nal theoretic methods [1]. Structural methods are based on a
more or less deterministic arrangement of textural elements
(texels). They are mainly used in industrial quality control,
where artical patterns are regular and dierences between
model and reality indicate failures [2,3]. Statistical meth-
ods dene textures as stochastic processes and characterize
them by a few statistical features. Most relevant statistical
approaches are Co-occurrence matrices [4], Markov random
elds [5] and autocorrelation methods [6]. Signal theoretic
approaches focus on periodic pattern resulting in peaks in
the spatial frequency domain, e.g. Gabor ltering [7,8] and
wavelet decomposition [9].
1
Christoph Palm is now with the Aixplain AG, Monheimsallee
22, D-52062 Aachen, Germany.
E-mail address: christoph.palm@web.de (C. Palm).
Beside texture, color is an important issue not only in hu-
man vision but in digital image processing where its impact
is still rising. In contrast to intensity, coded as scalar gray
values, color is a vectorial feature assigned to each pixel in
a color image. The mathematical dierence between scalars
and vectors for gray and color values, respectively, demands
a careful transfer of methods from the gray-scale to the color
domain. Although the use of color for texture analysis is
shown to be advantageous, the integration of color and tex-
ture is still exceptional. We dierentiate between parallel,
sequential and integrative methods (Section 2).
Several studies favor the statistical texture modeling and
in particular Co-occurrence matrices in the gray-scale do-
main [10]. Nevertheless, just few approaches were made
to transfer Co-occurrence matrices to color images [35]. In
this paper, we study existing approaches for color texture
classication by Co-occurrence features and propose novel
matrices and features to allow the exploitation of the color
information.
After the introduction of general color texture con-
cepts (Section 2) and details of the most relevant color
spaces (Section 3), we shortly repeat the concept of
0031-3203/$30.00 ? 2003 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.
doi:10.1016/j.patcog.2003.09.010
966 C. Palm/ Pattern Recognition 37 (2004) 965976
Co-occurrence matrices for gray-scale images (Section 4).
This concept is extended to the color domain (Section
5) dening single- and multi-channel matrices, respec-
tively. The description of the experimental setup (Section
6) is followed by the results and a detailed discussion
(Section 7).
2. Combining color and texture
We group the approaches combining color and texture
into parallel, sequential and integrative approaches.
The parallel concept for color texture analysis separates
the processing of both phenomena. Color is measured glob-
ally according to the histogram ignoring local neighboring
pixels. Texture is characterized by the relationship of the in-
tensities of neighboring pixels ignoring their color. The re-
sults of both analyses are combined subsequently to a feature
vector (Fig. 1, top). The parallel approach is advantageous,
because the known methods of gray-scale texture analysis
as well as pixel based color analysis can be applied directly.
Up to now, parallel algorithms are most commonly used for
image retrieval applications with color and texture as sep-
arated discriminative features [11]. However, the view on
texture as a pure intensity based structure is very simplied
and disregards, for instance, colored texture primitives with
constant intensity.
Sequential approaches use color analysis as a rst step of
a process chain. Clustering the color histogram, a partition-
ing of the image is obtained. These consecutive numbered
segments contain color information, but they are just repre-
sented by their indices and, hence, by scalars. Subsequently,
the index pattern is processed as gray-scale texture (Fig. 1,
middle). The sequential approach is related to the structural
texture model, where the pattern is composed by (color)
primitives. Hauta-Kasari et al. used it for industrial qual-
ity control [12], Song et al. especially for defect detection
in granite images [13], and Kukkonen et al. for inspection
of ceramic tiles [14]. These examples show the usefulness
of the sequential approach. Nevertheless, it is based on the
segmentation procedure, which depends on several parame-
ters and, therefore, gives no reproducible results. Addition-
ally, the assumption of colored texture primitives cannot be
generalized.
The third way of combining the analysis of color and
texture is called integrative, because the information de-
pendency between both image features is taken into ac-
count. Integrative methods can be further dierentiated into
single- and multi-channel strategies (Fig. 1, bottom). The
single-channel methods apply the gray-scale texture anal-
ysis on each color channel separately. The color informa-
tion is used indirectly restricting the intensity pattern to the
wavelength interval associated with the color channel. The
advantage of the single channel approach is the easy adap-
tion of known methods based on the gray-scale domain.
Recently, some of these were compared with the paral-
grayscale image
color image
analysis
color histogram
feature
feature
analysis
segmen-
analysis tation
color image color histogram label image
feature
color image
feature
analysis
feature
analysis
analysis
feature
feature
analysis
feature
analysis
analysis
feature
single channel
multi channel
color channels
Fig. 1. Top: Illustration of the parallel concept for color texture
analysis. Textural features and histogram features are strictly sep-
arated. Middle: Illustration of the sequential concept for color tex-
ture analysis. After the color segmentation of the histogram the
texture features are determined on basis of the segment indices.
Bottom: Illustration of integrative single- and multi-channel color
texture analysis. Both methods take color and texture into account.
lel concept and showed signicantly improved results [15].
Integrative multi-channel texture analysis handles two (or
more) channels simultaneously. These approaches have al-
ready been proposed for well known textural features like
Markov random elds [16], wavelet [17], Gabor lters [18
21], and autocorrelation features [22], showing encouraging
results.
Up to now, Co-occurrence matrices are just adapted for
sequential [12,14] and integrative single-channel color tex-
ture analysis [15]. The idea of multi-channel Co-occurrence
matrices was rstly addressed by Rosenfeld et al. in 1982,
but found to be computationally cumbersome [23]. In this
paper, we rstly extend the Co-occurrence approach to
C. Palm/ Pattern Recognition 37 (2004) 965976 967
integrative multi-channel color texture features. Addition-
ally, we introduce an information measure to describe the
supplementary information content of the multi-channel in
comparison to the single-channel approach. The extensive
experimental evaluation of the features in a classication
task takes two color spaces into consideration. The results
show the advantages of the multi-channel approach and
demonstrate the existence of non-intensity based pure color
textures.
3. Color spaces
To study the relation of intensity textures, pure color tex-
tures and combinations of both, respectively, the represen-
tation of intensity within the color space is important. The
RGB (red, green, blue) color space is a projection of the
cumulated intensity of reected light within a dened range
of wavelength to the axes of a three-dimensional space.
Certainly, the RGB values are not only dependent on the
reectance properties of the observed pattern but on the il-
luminant and the recording characteristics of the camera.
The problems of color constancy and device independency
are studied in detail elsewhere [22,24] and are not topic of
this paper. Because of the wavelength-dependent intensity
cumulation each color band is inherently a combination of
color and intensity. Additionally, a strong correlation be-
tween the color channels is reported [25].
De-correlating the RGB color bands and separating color
and intensity yields the LUV color space [26]:

L
U
J

= H
LUJ

R
G
B

=
1

2

2

2
2 1 1
0

3

R
G
B

. (1)
Tan et al. reported improving results for color histogram
analysis using LUV [26]. Independently from the idea of
mathematical de-correlation, a similar color space is dened
modeling human opponent color vision [27]. Additionally,
the LUV space is strongly related to the complex color space
since the color planes spanned by UJ and the complex col-
ors are equal [28]. With RGB and LUV we study color
spaces where intensity and color are combined or separated,
respectively.
4. Gray-scale Co-occurrence matrices
In contrast to histograms, Co-occurrence matrices
(CMs) characterize the relationship between the values of
d
Fig. 2. Illustration of the concept of gray-scale Co-occurrence ma-
trices. The Co-occurrence matrices within one octant are combined
to one mean meatix to be more robust for long distances d.
neighboring pixels [4]. Therefore, they represent a second
order statistical measurement. We denote the values of a
gray-scale image [ as w{0, . . . , W 1} with [(p) = w.
The pixel position is given by p =(m, n) with m, n Z. The
distance vector d is noted in polar coordinates (d, p) with
discrete length and orientation d, p N, respectively, com-
puted from linear coordinates applying the polar transform
and an appropriate truncation. The probability Pr of two val-
ues w and w co-occurring with pixel positions related by d,
denes the cell entry (w
1
, w
2
) of the CM C
d
:
C
d
(w, w)
:= Pr([(p
1
) =w [(p
2
)
= w| |p
1
p
2
| = d). (2)
Hence, C
d
is a symmetric two-dimensional neighborhood
histogram, which depends on d. To study the eect of
changing d to the classication results, we extend C
d
to
long-distance CMs. Fig. 2 visualizes the idea to build the
mean CM for each octant of a discrete circle approxima-
tion. The single CMs within the octant are summed up and
normalized keeping d constant. Because of the symmetric
denition of CM only four mean CMs remain. Neverthe-
less, with (W W) they are high dimensional. To reduce
the large amount of data represented by CMs, two methods
are applicable:
quantize the image into few values as a preprocessing step
extract special features to describe CM data as a
post-processing method
In this work we applied the last approach and used the matrix
features introduced by Haralick et al. [4]. Unfortunately,
some of the 14 features show redundancies, but it is not yet
clear, which of them can be ignored. Therefore, we selected
eight features (homogeneity, contrast, correlation, variance,
inverse dierence moment, entropy, correlation I, correlation
968 C. Palm/ Pattern Recognition 37 (2004) 965976
II), that build the set union used in [10,15,29]. They are well
distributed over the four feature groups dened in [10].
The Cartesian product for the selected Haralick features
generates a (8 4)-space for the rotationally variant CM
features. To be rotationally invariant, the mean and the vari-
ance of the orientation-dependent features are determined
separately. Overall, the gray-scale CM features (GCFs) are
vectors of size 8 2.
5. Integrative color Co-occurrence matrices
According to our categorization (Section 2), in the follow-
ing two integrative extensions of gray-scale CMs to color
CMs are proposed. The single-channel Co-occurrence ma-
trices (SCMs) consist of gray-scale CMs successively ap-
plied to separated color channels. The correlation between
textures of dierent color channels is captured by novel
multi-channel Co-occurrence matrices (MCMs).
5.1. Single-channel Co-occurrence matrices
5.1.1. Background
For the denition of SCMs, we assume a limited
K-dimensional Cartesian space like RGB and LUV with
K = 3. A SCM SC
k
d
corresponds to C
d
applied to the kth
color channel [
k
of the image f = ([
1
, . . . , [
K
)
T
:
Pr(w
k
, w
k
)
:= SC
k
d
(w, w)
:= Pr([
k
(p
1
) =w [
k
(p
2
) = w| |p
1
p
2
| = d). (3)
The joint probability Pr(w
k
, w
k
) indicates the adjacency
of w and w on the same channel k. The corresponding
rotational invariant single-channel Co-occurrence features
(SCFs) consist of K feature vectors SCF
k
, where SCF
k
is
analog to GCF according to k: SCF = SCF
1
SCF
K
.
Following Section 3, RGB combines intensity as well
as color information. Therefore, RGB-based SCFs contain
combined color and intensity texture information. Recently,
Drimbarean et al. showed RGB-based SCF-like features to
outperform GCFs [15]. In contrast to RGB, LUV separates
intensity and color strictly. Hence, SCF
1
represents pure
intensity texture and equals GCF. In contrast, pure color
textures are modeled by SCF
2
and SCF
3
according to the
channels U and J, respectively.
5.1.2. Information measurement
The quantication of the information prot of CMs in
comparison to common histograms is based on the Kol-
mogorov distance [30], which measures the dierence be-
tween two probability distributions. The one-dimensional
histogram Pr(w
k
) for each channel k is derived from the
Image Number
D
i
s
t
a
n
c
e
1
0 30
0.2
0.8
1
Image Number
D
i
s
t
a
n
c
e
0 30
0.2
0.8
Image Number
D
i
s
t
a
n
c
e
1
0 30
0.2
0.8
Image Number
D
i
s
t
a
n
c
e
1
0 30
0.2
0.8
Fig. 3. Top left: Kolmogorov distance D(SC
k
d
) for single-channel
Co-occurrence matrices with k = R (white), k = G (black),
and k = B (gray). The image numbers follow the ascending
order of D(SC
G
d
). Bottom left: D(SC
k
d
) with k = L (black)
k = U (gray), and k = J (white). The image numbers fol-
low the ascending order of D(SC
L
d
). Top right: D(MC
k
1
, k
2
d
)
for multi-channel Co-occurrence matrices with (k
1
, k
2
) = (R, G)
(white), (k
1
, k
2
) =(R, B) (black), and (k
1
, k
2
) =(G, B) (gray). The
image numbers follow the ascending order of D(MC
R, B
d
). Bottom
right: D(MC
k
1
, k
2
d
) with (k
1
, k
2
) = (U, J) (black) (k
1
, k
2
) = (L, U)
(gray), and (k
1
, k
2
) = (L, J) (white). The image numbers follow
the ascending order of D(MC
U, J
d
).
corresponding SC
k
d
by summing up the columns:
Pr( w
k
) =
W
k
1

w=0
SC
k
d
(w, w). (4)
In the case of stochastic independency of d-adjacent values,
the joint probability

Pr(w
k
, w
k
) is estimated by the multi-
plicative combination of the univariate densities Pr(w
k
) and
Pr( w
k
):

Pr(w
k
, w
k
) = Pr(w
k
) Pr( w
k
). (5)
In contrast, the actually measured density Pr(w
k
, w
k
) is re-
lated to the conditional probability Pr(w
k
| w
k
):
Pr(w
k
, w
k
) = Pr(w
k
| w
k
) Pr( w
k
). (6)
The Kolmogorov distance D(SC
k
d
) as dierence between

Pr(w
k
, w
k
) and Pr(w
k
, w
k
) is equal to the dierence between
random adjacency and the actual adjacency of w and w.
Therefore, it is a measure of stochastic dependency and,
consequently, a measure of information gain of the CM in
contrast to the univariate histogram:
D(SC
k
d
) =
1
2
W
k
1

w=0
W
k
1

=0
|Pr(w
k
, w
k
)

Pr(w
k
, w
k
)|. (7)
With the help of D(SC
k
d
), the general information prot of
CMs specic to the color channels is determined. Fig. 3
(left) shows the Kolmogorov distance values of 30 images
of the VisTex database (see Section 6) corresponding to
RGB and LUV.
C. Palm/ Pattern Recognition 37 (2004) 965976 969
According to RGB, D(SC
k
d
) ranges from 0.2 to 0.8. Com-
pared to the baseline of zero (no information prot), this
range shows the benet of texture analysis by CM in con-
trast to pure histogram analysis. Obviously, the Kolmogorov
distances of R, G and B are correlated. Nevertheless, for
single textures dierences up to 0.2 between two of them
have to be recognized. This indicates the non-redundancy of
channel-specic texture measures in the RGB color space.
Similar to RGB, D(SC
k
d
) for LUV range from 0.1 to 0.8.
Especially the curve according to L show noticeable par-
allels to the RGB related curves. This indicates the strong
intensity relation of the RGB color channels. In contrast
to RGB, only the Kolmogorov distances for U and J are
strongly correlated, but they show signicant dierences to
the L values. Hence, the information prot of pure color
texture analysis is dierent from the information prot of
intensity-based texture analysis. Most frequently, with val-
ues between 0.2 and 0.7 the information prot by intensity
texture recognition is higher than that for pure color tex-
ture analysis. Nevertheless, comparing to the zero baseline
pure color texture features are proved to provide additional
information of substance for color texture analysis.
5.2. Multi-channel Co-occurrence matrices
5.2.1. Background
The SCMs ignore the correlation between color chan-
nels, although it may provide additional textural information
[16]. Taking this into account, we introduce multi-channel
Co-occurrence matrices (MCMs) MC
k
1
, k
2
d
, which count the
pairwise occurrence of values in dierent bands k
1
and k
2
with k
1
= k
2
:
Pr(w
k
1
, w
k
2
)
:= MC
k
1
, k
2
d
(w, w)
:= Pr([
k
1
(p
1
) =w [
k
2
(p
2
) = w| |p
1
p
2
| = d). (8)
For MCMs the symmetry is lost. Since (MC
k
1
, k
2
d
)
T
=
(MC
k
2
, k
1
d
) is valid obviously, the mean matrix of (MC
k
1
, k
2
d
)
and (MC
k
2
, k
1
d
) is symmetric again, which results in K
2
K
dierent mean matrices. Again, Haralick-features are ap-
plied on the mean MCMs to extract 16(K
2
K)}2 rotational
invariant multi-channel Co-occurrence features (MCFs).
The interpretation of MCF in the LUV color space has
to dierentiate between intensity dependent and indepen-
dent parts, respectively. Whereas the MCF for (k
1
, k
2
) =
(U, J) represents pure color neighborhood, the L-related
MCF (k
1
, k
2
) =(L, U) and (k
1
, k
2
) =(L, J) are intensity de-
pendent and characterize the correlation of color and inten-
sity with regard to texture.
Assuming the spatial distance vector d =0, the represen-
tation of MCM in Eq. (8) is related to the two-dimensional
color histogramapproximation, because no spatial adjacency
but channel adjacency is regarded. For this, histogram fea-
tures HCFs can be extracted directly from the MCM. The
color histogram distribution is here noted as Pr
0
(w
k
1
, w
k
2
).
Color histograms are frequently used to characterize the
image color globally without considering local texture in-
formation. Hence, MCM provide the opportunity to study
the eect of texture and pure color analysis in one unied
framework.
5.2.2. Information measurement
In Section 5.1.2 we introduced the Kolmogorov distance
as a measure of the information gain using Co-occurrence
matrices instead of histograms. After denition of MCMs,
this measure is now extended to quantify the relation of
color and texture for each image objectively.
In analogy to Eq. (5), the adjacency of two values w and w
across two color channels, k
1
and k
2
, is a multiplicative com-
bination of the color histogramdistribution Pr
0
(w
k
1
, w
k
2
) and
the channel-specic distribution Pr( w
k
2
, w
k
2
). The product
is normalized by Pr( w
k
2
) and summed up over all possible
w
k
2
:

Pr(w
k
1
, w
k
2
)
=

wW
1
Pr( w
k
2
)
Pr
0
(w
k
1
, w
k
2
) Pr( w
k
2
, w
k
2
). (9)
Note, that Eq. (9) assumes the stochastic independency of
Pr
0
(w
k
1
, w
k
2
) and Pr( w
k
2
, w
k
2
). The parallel color texture
concept (Section 2) is based on this independency. The es-
timated distribution

Pr(w
k
1
, w
k
2
) in Eq. (9) is computable
using SC
k
d
and MC
k
1
, k
2
0
.
For the actual distribution Pr(w
k
1
, w
k
2
) conditional prob-
abilities are needed:
Pr
0
(w
k
1
, w
k
2
)
=

w
1
Pr( w
k
2
)
Pr(w
k
1
, w
k
2
| w
k
2
, w
k
2
) Pr( w
k
2
, w
k
2
).
(10)
The Kolmogorov distance D(MC
k
1
, k
2
d
) quanties the infor-
mation prot of multi-channel color texture analysis in con-
trast to the parallel concept by separated analysis of color
and texture:
D(MC
k
1
, k
2
d
)
=
1
2
W
k
1
1

w=0
W
k
2
1

w=0
|Pr(w
k
1
, w
k
2
)

Pr(w
k
1
, w
k
2
)|
=
1
2
W
k
1
1

w=0
W
k
2
1

w=0
W
k
2
1

w=0
Pr( w
k
2
, w
k
2
)
Pr( w
k
2
)
(|Pr
0
(w
k
1
, w
k
2
) Pr
0
(w
k
1
, w
k
2
| w
k
2
, w
k
2
)|). (11)
D(MC
k
1
, k
2
d
) reaches zero, if either the texture image is
non-colored, (identical red, green and blue channel), or
the color image is non-textured (each pixel value has the
same neighbor). In each case, MCMs provide no additional
information to SCMs and histograms, respectively.
970 C. Palm/ Pattern Recognition 37 (2004) 965976
Fig. 3 (right) shows D(MC
k
1
, k
2
d
) for the natural color
textures of the VisTex database (see Section 6) for RGB
and LUV, respectively. According to RGB, the values
range from 0.1 to 0.7. Considering the baseline of zero for
stochastic independency of color and texture, this range
indicates clearly the high information prot using the novel
multi-channel Co-occurrence features in contrast to any
parallel method. The similarity of D(MC
k
1
, k
2
d
) for dierent
color channels regarding to each image separately allows
their interpretation as a global color texture feature rather
than a channel-specic feature.
In contrast to RGB, the curves for dierent channel com-
binations of LUV vary signicantly. Whereas D(MC
k
1
, k
2
d
)
for the intensity independent combination (k
1
, k
2
) = (U, J)
reaches values in [0.7, 1.0], the distances according to the
combinations of color and intensity channels are mostly less
than 0.3. Therefore, the information prot analyzing inten-
sity independent color textures is very high and the existence
of pure intensity independent color texture obvious.
6. Experimental evaluation
Studying the Kolmogorov distance, we already showed
the existence of intensity independent color textures as well
as the usefulness of the integrative color texture analysis.
These results will be conrmed by experimental evaluation.
6.1. Experimental setup
To allow the generalization of the classication results,
two dierent image sets are used. From the VisTex dataset
[31] of MIT we choose 30 images of size 512512 (Fig. 4),
each of which contains one natural colored texture. One tex-
ture class is built splitting one image into 64 non-overlapping
sub-images of size 64 64. Therefore, inter-class variation
corresponds to local variation of the full-size images. This
dataset is here addressed by DS
1
. The second dataset called
DS
2
contains images of natural barks [32], built up in the
context of the PhD thesis of Lakmann [33]. The images con-
sist of one type of bark and background. Each image shows
a dierent tree, which is previously classied into one of six
classes (Fig. 5). Therefore, the inter-class variation reects
the natural surface aberration of trees of the same kind. Each
class consists of 68 images of size 384 256 yielding a
collection of 408 images. Since we apply an image classi-
cation not a texture segmentation, the image border is ex-
cluded dening a Region-of-Interest of xed size 300200
located at the image center.
For classication, the leaving-one-out method is used to
guarantee strict separation of test and training set in com-
bination with the maximization of the number of training
images. A 5-Nearest-Neighbor classier using the Euclid-
ian vector distance was employed. To achieve a compara-
ble range of values for dierent features, a normalization
Fig. 4. Images of the VisTex database (leftright, updown): Bark0,
Bark4, Bark6, Bark8, Bark9, Brick1, Brick4, Brick5, Fabric0, Fab-
ric4, Fabric7, Fabric9, Fabric11, Fabric13, Fabric16, Fabric17, Fab-
ric18, Food0, Food2, Food5, Food8, Grass1, Sand0, Stone4, Tile1,
Tile3, Tile7, Water6, Wood1, Wood2 [31].
Fig. 5. Images of the BarkTex database, one example for each
class (leftright, updown): birch, beech, spruce, pine, oak,
robinia [32].
C. Palm/ Pattern Recognition 37 (2004) 965976 971
Table 1
The notation dened in this table will be needed for understanding the following tables of classication results
Space Feature vector Abbreviation E
gray GCF m(L) 16
RGB SCF m(R), m(G), m(B) 16
RGB MCF m(RG), m(RB), m(GB) 16
RGB HCF m(RG
0
), m(RB
0
), m(GB
0
) 8
LUV SCF m(L), m(U), m(J) 16
LUV MCF m(LU), m(LJ), m(UJ) 16
RGB m(R) m(G) m(B) m(RGB
SC
) 48
RGB m(RG) m(RB) m(GB) m(RGB
MC
) 48
RGB m(RGB
SC
) m(L) m(RGB
SC
, L) 64
RGB m(RGB
MC
) m(L) m(RGB
MC
, L) 64
RGB m(RG
0
) m(RB
0
) m(GB
0
) m(RGB
0
) 24
RGB m(RGB
0
) m(L) m(RGB
0
, L) 72
RGB m(RGB
0
) m(RGB
SC
) m(RGB
0, SC
) 72
RGB m(RGB
0
) m(RGB
MC
) m(RGB
0, MC
) 96
RGB m(RGB
SC
) m(RGB
MC
) m(RGB
SC, MC
) 96
RGB m(RGB
0
) m(RGB
SC
) m(RGB
MC
) m(RGB
0, SC, MC
) 120
LUV m(U) m(J) m(UJ) m(UJ
II
) 48
LUV m(LU) m(LJ) m(L) m(LUJ
ID
) 48
LUV m(UJ
II
) m(LUJ
ID
) m(LUJ
II, ID
) 96
The gray-scale Co-occurrence features, single-channel Co-occurrence features, multi-channel Co-occurrence
features and histogram Co-occurrence features are here noted as the feature vectors GCF, SCF, MCF and HCF,
respectively. The abbreviations of the feature vectors m make clear, which color channels are involved. These
vectors are combined by Cartesian product . The indices MC, SC and 0 indicate multi-channel Co-occurrence,
single-channel Co-occurrence and histogram features, respectively, in the RGB color space. For LUV, the
feature vectors are combined to intensity independent (II) and intensity dependent vectors (ID). E symbolizes
the dimension of the feature space.
according to variance o
e
and mean j
e
of each dimension e
is done separately exploiting the training data 1R:
m

e
=
m
e
j
e
o
e
(12)
with
j
e
=
1
M

m1R
m
e
, o
e
=

1
M 1

m1R
(m
e
j
e
)
2
. (13)
The eth element of an E-dimensional feature vector m and
the corresponding normalized element is denoted with m
e
and m

e
, respectively.
The optimal parameter d of a CM depends on the resolu-
tion of the texture. Zucker et al. developed a method to de-
termine this optimal distance [34]. However, this optimum
refers to a distinct texture and one single color channel. In
best case, it can be generalized to the entire texture class. In
our classication experiments several classes are involved.
Hence, an optimal distance for all color channels and all
classes cannot be determined. Consequently, we studied the
color textures according to several distances. The follow-
ing experiments are performed with d = 0 (histogram) and
d = 1, 5, 10, 15, 20.
We combined the previously mentioned SCFs and MCFs
and compared them to pure gray-scale texture and pure color
histogram classication results. To give a clear reference to
the feature vectors m used in the experiments, the abbrevi-
ations of the single- and combined feature vectors are listed
in Table 1.
Note, that the combinations in LUV are grouped into
intensity independent (II) and intensity dependent (ID)
whereas in RGB they are grouped into SCFs (SC) and
MCFs (MC).
6.2. Results
The classication results for DS
1
and DS
2
are listed in
Tables 2 and 3, respectively. The CM-representation of the
color histogram allows the direct comparison of the parallel
and the integrative color texture concept (Section 2). Rows
13 of both tables show the results of gray-scale texture fea-
tures m(L) and the color histogram, described by its Haral-
ick features m(RGB
0
). Obviously, the importance of color
and intensity texture is quite dierent for the two datasets.
Whereas the HCFs for DS
1
performs with 0.938 better than
the GCFs with 0.855, the texture features show better dis-
crimination eciency with 0.787 compared to 0.654 for
m(RGB
0
) according to DS
2
. For both datasets the parallel
concept combining the two single features outperform with
0.977 and 0.838 for DS
1
and DS
2
, respectively, the results
of the single features.
972 C. Palm/ Pattern Recognition 37 (2004) 965976
Table 2
Classication results for dataset DS
1
according to dierent color texture feature combinations
DS
1
0 1 5 10 15 20
1 m(L) 0.855 0.652 0.544 0.531 0.518
2 m(RGB
0
) 0.938
3 m(RGB
0
, L) 0.977 0.945 0.929 0.910 0.928
4 m(R) 0.868 0.771 0.726 0.702 0.691
5 m(G) 0.869 0.779 0.697 0.691 0.677
6 m(B) 0.896 0.819 0.738 0.720 0.704
7 m(RGB
SC
) 0.951 0.884 0.826 0.811 0.793
8 m(RGB
MC
) 0.960 0.870 0.862 0.850 0.864
9 m(RGB
SC, MC
) 0.971 0.903 0.887 0.864 0.869
10 m(RGB
0, SC
) 0.974 0.951 0.928 0.906 0.913
11 m(RGB
0, MC
) 0.972 0.933 0.920 0.925 0.926
12 m(RGB
0, SC, MC
) 0.977 0.946 0.928 0.917 0.920
13 m(RGB
SC
, L) 0.946 0.868 0.821 0.807 0.804
14 m(RGB
MC
, L) 0.966 0.894 0.865 0.864 0.882
15 m(RGB
SC, MC
, L) 0.968 0.902 0.905 0.876 0.879
16 m(U) 0.886 0.754 0.666 0.609 0.595
17 m(J) 0.856 0.719 0.592 0.536 0.532
18 m(UJ) 0.891 0.799 0.795 0.794 0.661
19 m(UJ
II
) 0.969 0.910 0.867 0.840 0.843
20 m(LUJ
ID
) 0.948 0.895 0.870 0.852 0.863
21 m(LUJ
II, ID
) 0.986 0.949 0.923 0.916 0.922
The dierent results in one row refer to CM distances from zero (color histogram) up to 20. The feature
combinations are divided into semantically interrelated blocks: parallel color texture concept (Rows 13),
integrative color texture concept in the RGB color space (Rows 49), combination of integrative features with
pure color and pure intensity texture features, respectively (Rows 1015) integrative color texture concept in
the LUV color space (Rows 1621).
The results for SCF of the RGB color space are presented
in Rows 46. The best results are obtained by m(B) with
d = 1 and correctness 0.896 and by m(B) with d = 5 and
correctness 0.794 for DS
1
and DS
2
, respectively. A general
superior performance of the B channel is not plausible and
depends on the image dataset. All single results for both
datasets are superior to the corresponding pure gray-scale
texture result. These classication rates for m(R), m(G),
m(B) are improved signicantly (unidirectional Binomial
test statistic, 0.05) by combining them to m(RGB
SC
)
(Row 7). According to m(RGB
SC
) we achieved a correct-
ness of 0.951 and 0.828 for DS
1
and DS
2
, respectively. The
corresponding combination of MCFs to m(RGB
MC
) yield
with 0.960 better and with 0.795 even worth results for DS
1
and DS
2
. Combining m(RGB
SC
) and m(RGB
MC
) (Row 8),
the dimension of the feature space rises to 96 and, hence,
the complexity of classication task rises, too. Nevertheless,
the results improved to 0.971 and 0.841, respectively.
The results of Rows 1012 in Tables 2 and 3 allow the
evaluation of possible advantages combining RGB-related
integrative color texture features with the pure color his-
togram features. For DS
1
the combination of SCF, MCF and
both with histogram features showed slightly increasing re-
sults, respectively. The overall combination m(RGB
0, SC, MC
)
performed best with a correctness of 0.977. According to
DS
2
the combination of SCF with histogram features im-
prove the performance of SCF with a correctness of 0.841.
In contrast, the performance of the overall combination de-
creased to 0.808. However, the feature space dimension and,
therefore, the complexity of the classication problem, rose
to 120 for this combination (see Table 1). All results of tex-
ture feature combination with the histogram features outper-
form the pure histogram features.
Rows 1315 show the classication results combining
RGB-based SCFs and MCFs with GCFs. For both datasets
all color texture combinations performed better than the pure
C. Palm/ Pattern Recognition 37 (2004) 965976 973
Table 3
Classication results of dataset DS
2
according to dierent color texture feature combinations
DS
2
0 1 5 10 15 20
1 m(L) 0.721 0.787 0.742 0.694 0.637
2 m(RGB
0
) 0.654
3 m(RGB
0
, L) 0.831 0.838 0.814 0.765 0.767
4 m(R) 0.723 0.775 0.708 0.664 0.639
5 m(G) 0.745 0.777 0.740 0.689 0.627
6 m(B) 0.740 0.794 0.750 0.703 0.708
7 m(RGB
SC
) 0.787 0.828 0.784 0.708 0.666
8 m(RGB
MC
) 0.807 0.795 0.747 0.705 0.738
9 m(RGB
SC, MC
) 0.809 0.841 0.789 0.767 0.746
10 m(RGB
0, SC
) 0.826 0.841 0.806 0.794 0.779
11 m(RGB
0, MC
) 0.812 0.798 0.759 0.764 0.746
12 m(RGB
0, SC, MC
) 0.824 0.808 0.813 0.803 0.795
13 m(RGB
SC
, L) 0.777 0.824 0.784 0.703 0.668
14 m(RGB
MC
, L) 0.809 0.797 0.767 0.794 0.733
15 m(RGB
SC, MC
, L) 0.804 0.847 0.772 0.762 0.768
16 m(U) 0.658 0.776 0.714 0.646 0.595
17 m(J) 0.653 0.614 0.597 0.585 0.548
18 m(UJ) 0.532 0.548 0.484 0.548 0.453
19 m(UJ
II
) 0.756 0.807 0.759 0.778 0.729
20 m(LUJ
ID
) 0.786 0.740 0.694 0.729 0.673
21 m(LUJ
II, ID
) 0.863 0.806 0.836 0.856 0.786
The notation corresponds to that of Tables 1 and 2.
gray-scale texture features. The combined features show
similar results as the corresponding single features in Rows
79. Therefore, no tendency can be remarked which indi-
cates the usefulness of these combinations. For both datasets
the best result of this block is achieved by the overall com-
bination m(RGB
SC, MC
, L). It yields a correctness of 0.968
and 0.847 for DS
1
and DS
2
, respectively.
The results related to the intensity independent single
feature vectors of the LUV color space are listed in Rows
1618. For DS
1
the best performance is achieved with 0.891
by m(UJ) and d = 1. On the other hand, for DS
2
the best
correctness of 0.776 is achieved by m(U) and d=5. There-
fore, both are below the respective maxima according to the
single RGB features (Rows 46).
For both datasets the results are enhanced by combin-
ing the single features to m(UJ
II
) (Row 19). With 0.969
and 0.807 for DS
1
and DS
2
, respectively, the improvement
is signicant (0.05). The intensity independent texture
features are in both cases superior to the intensity depen-
dent features (Row 20). Note, that m(L) is inherently in-
tegrated into m(LUJ
ID
) as SCF
1
of the LUV color space.
The best results are obtained by the combination of both, in-
tensity independent and intensity dependent features (Row
21). These are superior not only to other combinations of
the RGB color space but to the parallel color texture con-
cept m(RGB
0
, L) as well. The results 0.986 and 0.863 for
DS
1
and DS
2
, respectively, refer both to d = 1.
Taking d into account, a strong tendency to an opti-
mum distance for each dataset can be stated. All supe-
rior results for each experiment block are found for d = 1
and (except one) for d = 5 according to DS
1
and DS
2
,
respectively.
7. Discussion
In our experiments, gray-scale texture and color histogram
analysis were compared with novel color texture features.
Additionally, two dierent concepts of color texture analy-
sis, parallel and integrative, were evaluated. This was possi-
ble for the rst time, because in the framework of CMs color
histograms and textural features are both well described.
The two image datasets used are rather dierent in content,
color texture appearance as well as discrimination capabil-
ity of color histogram and gray-scale texture. Nevertheless,
the tendencies of the results are quite similar. Therefore,
974 C. Palm/ Pattern Recognition 37 (2004) 965976
it is possible to generalize them to large application elds
dealing with natural color textures.
It has been shown, that color information improve the re-
sults of gray-scale texture features [14]. Our experiments
support these results. The combination of color histograms
and gray-scale texture features shows a high supplementa-
tion ability. The results of this parallel color texture concept
are just slightly below the best results we achieved by inte-
grative color texture features.
Whereas the parallel concept is most commonly applied
in the color image processing community, this paper intro-
duced novel integrative color texture features to learn more
about the aspects of color texture and develop new meth-
ods to model them. According to SCMs, we realize notice-
able dierences between the single-channel results. This in-
dicates dierences in the textures of dierent color channels
and relativizes their correlation. This observation supports
our evaluation of Kolmogorov distances in Section 5. Addi-
tionally, the channel-specic features supplement each other
resulting in a signicant better performance for the combi-
nation features. This enhancement is noticeable in particu-
lar considering the increase of dimensionality of the feature
space. Because the gray-scale texture results are below all
single-channel results, the transform from color to gray val-
ues is lossy with respect to textural features.
The novel method of MCMs improve the results of the
SCMs taking the correlation between color channels into ac-
count. With the help of the Kolmogorov distance, we demon-
strated that the multi-channel modeling is not equivalently
realized by the single-channel features in combination with
the color histogram. Therefore, MCMs measure new color
textural features. The low redundancy of both integrative
methods is also shown by the experimental evaluation. The
combination of m(RGB
SC
) and m(RGB
MC
) leads to increas-
ing correctness although the feature space dimensionality
rises by a factor of two.
The combination of the SCFs with the histogram features
show rather similar results to the parallel color texture con-
cept. This could be expected since the only dierence is the
already stated wavelength dependency of colored texture.
Unfortunately, the information prot of texture modeling
subsequently on the color channels is over-compensated by
the disproportionate extension of the feature space. Never-
theless, color histogram and single-channel texture analysis
supplement each other. Therefore, the classication correct-
ness rises in comparison to both single components. On the
other hand, the combination of m(RGB
SC
) with the GCFs
yields worse results according to both components. Again,
this shows the high redundancy of GCFs and SCFs. Study-
ing the redundancy of MCFs with HCFs and GCFs, re-
spectively, in both cases better results in comparison with
the single components can be stated. Therefore, a new kind
of texture information is measurable by the multi-channel
analysis.
In addition to RGB, the experimental evaluation take
the LUV color space into account. Although the methods
of single- and multi-channel color texture analysis are the
same, the interpretation of the results has to be adapted.
Because of the two intensity independent channels U and
J we achieve three intensity dependent and three intensity
independent matrices. Obviously, the intensity independent
features model a completely dierent kind of texture. Al-
though the RGB related color texture features consider both
color and texture, the texture is again intensity based, even
though this intensity is measured on a small wavelength in-
terval. In contrast, the MCMs related to U and J charac-
terize the Co-occurrence of pure colors. Because the three
single feature vectors m(U), m(J) and m(UJ) represent
dierent parts of the color plane, they supplement each other
and, hence, their combination yields to a signicant better
classication result. Additionally, the results to the intensity
independent features are better than that of the intensity de-
pendent. The novel pure color texture features in compari-
son with pure intensity texture features show the overall best
performance. Again, this proves the fact low redundancy
between these two concepts.
8. Conclusion
In this paper, we presented novel integrative single- and
multi-channel Co-occurrence matrices for color texture anal-
ysis and put intensity independent color textures in perspec-
tive.
The multi-channel approach integrates color histograms
into the Co-occurrence notation and allows the comparison
of parallel and integrative approaches on the same level
for the rst time. Additionally, the Kolmogorov distance
is recommended to measure quantitatively the information
gain of the multi-channel approach.
The advantages of the integrative methods were demon-
strated by the Kolmogorov distance as well as by the re-
sults of the experimental evaluation. Nevertheless, because
of the disproportional increase of feature space dimension-
ality, the best results according to RGB are achieved by the
parallel concept of separate analysis of color by histogram
and texture by intensity pattern analysis. However, the inte-
grative color features based on the LUV color space show
not only the existence of intensity independent color texture
but its discriminative power. In combination with the com-
mon intensity dependent pattern, colored textures are more
adequately described.
Summary
Color image analysis and texture classication are two
basic tasks in pattern recognition. The dependencies of
both are still not clear and, hence, several concepts dealing
with colored textures exist. The objective of this paper is
to strengthen the basis for the discussion in the eld of
color texture recognition. To achieve this goal, we treated
C. Palm/ Pattern Recognition 37 (2004) 965976 975
ve main elds within the framework of Co-occurrence
matrices:
Categorization of the concepts for color texture feature
extraction.
The identication of these concepts enables an overview
over the relevant literature and allows the comparison
between these approaches. Two of them, the parallel and
the integrative method, have been compared within the
experimental section.
Extension of Co-occurrence matrices to multi-channel
color texture characterization.
Since Co-occurrence matrices are one of the most fre-
quently used methods for gray-scale texture analysis,
multi-channel matrices enlarges the toolbox of color
texture analysis methods signicantly.
Application of Kolmogorov distance to the novel single-
and multi-channel matrices.
The Kolmogorov distance serves as a quantitative mea-
sure to determine the dependency of intensity textures and
colors. Therefore, the discussion about this dependency
in the scientic community can be adjudged objectively
for each image separately.
Application of integrative Co-occurrence matrices to the
RGB as well as the LUV color space.
Since the LUV space separates one intensity channel L
and two color channels U and V, Co-occurrence matri-
ces according to U and V provide intensity independent
and, hence, pure color texture features. This features are
used to prove the existence of intensity independent color
textures.
Classication experiments on two large image databases.
The large number of experimental data allows the gen-
eralization of the conclusions drawn on the basis of the
results. In particular, the parallel and the integrative con-
cept are compared in the framework of Co-occurrence
matrices.
Two main conclusions have to be emphasized. Firstly, the
parallel concept of intensity texture analysis and separately
color histogram analysis is suitable for color image classi-
cation. Secondly, the existence of pure color textures as well
as their supplementation ability with respect to intensity de-
pendent textures features is proven. Therefore, the integra-
tive color texture features introduced in this paper enable
the characterization of intensity independent color textures
for the rst time. Using these novel features in combina-
tion with gray-scale texture and color histogram features the
classication correctness is increased up to 32%.
References
[1] T.R. Reed, J.M.H. Du Buf, A review of recent texture
segmentation and feature extraction techniques, CVGIP
Image Understanding 57 (3) (1993) 359372.
[2] I. Tus, Automated fabric inspection based on a structural
texture analysis method, in: Recent Issues in Pattern Analysis
and Recognition, Springer, Berlin (1989) 377390.
[3] A. Bodnarova, M. Bennamoun, K.K. Kubik, Suitability
analysis of techniques for aw detection in textiles
using texture analysis, Pattern Anal. Appl. 3 (2000)
254266.
[4] R. Haralick, K. Shanmugam, I. Dinstein, Texture features
for image classication, IEEE Trans. Syst. Man Cybernet.
3 (1973) 610621.
[5] Lei Wang, Jun Liu, Texture classication using
multi-resolution markov random eld models, Pattern
Recognition Lett. 20 (2) (1999) 171182.
[6] M. Pietik ainen, T. Ojala, Z. Xu, Rotation-invariant texture
classication using feature distributions, Pattern Recognition
33 (2000) 4352.
[7] A.K. Jain, F. Farrokhnia, Un-supervised texture segmentation
using Gabor lters, Pattern Recognition 24 (12) (1991)
11671186.
[8] T. Randen, J.H. HusHy, Multichannel ltering for
image texture segmentation, Opt. Eng. 33 (8) (1994)
26172625.
[9] Jing-Wein Wang, Chin-Hsing Chen, Wei-Ming Chien,
Chih-Ming Tsai, Texture classication using non-separable
two-dimensional wavelets, Pattern Recognition Lett. 19 (13)
(1998) 12251234.
[10] C. Gotlieb, H. Kreyszig, Texture descriptors based on
Co-occurrence matrices, Comput. Vision Graph. Image
Process. 51 (1990) 4752.
[11] K. Messer, J. Kittler, A region-based image database system
using colour and texture, Pattern Recognition Lett. 20 (1999)
13231330.
[12] M. Hauta-Kasari, J. Parkkinen, T. Jaaskelainen, R. Lenz,
Multi-spectral texture segmentation based on the spectral
Co-occurrence matrix, Pattern Anal. Appl. 2 (4) (1999)
275284.
[13] K.Y. Song, J. Kittler, M. Petrou, Defect detection in
random colour textures, Image Vision Comput. 14 (1996)
667683.
[14] S. Kukkonen, H. K alvi ainen, J. Parkkinen, Color features for
quality control in ceramic tile industry, Opt. Eng. 40 (2)
(2001) 170177.
[15] A. Drimbarean, P.F. Whelan, Experiments in colour texture
analysis, Pattern Recognition Lett. 22 (2001) 11611167.
[16] P.-H. Suen, G. Healey, Modeling and classifying color
textures using random elds in a random environment, Pattern
Recognition 32 (1999) 10091017.
[17] G. Van de Wouwer, P. Scheunders, S. Livens, D. Van
Dyck, Wavelet correlation signatures for color texture
characterization, Pattern Recognition 32 (1999) 443451.
[18] A.K. Jain, G. Healey, A multi-scale representation including
opponent color features for texture recognition, IEEE Trans.
Image Process. 7 (1) (1998) 124128.
[19] C. Palm, D. Keysers, T.M. Lehmann, K. Spitzer, Gabor
ltering of complex hue/saturation images for color texture
classication, Joint Conference on Information Science 2,
Atlantic City, NJ, USA (2000) 4549.
[20] G. Paschos, Perceptually uniform color spaces for color texture
analysis: an empirical evaluation, IEEE Trans. Image Process.
10 (6) (2001) 932937.
976 C. Palm/ Pattern Recognition 37 (2004) 965976
[21] C. Palm, T.M. Lehmann, Classication of color textures by
Gabor ltering, Mach. Graph. and Vision 11 (2/3) (2002)
195219.
[22] G. Healey, L. Wang, Illumination-invariant recognition of
texture in color images, J. Opt. Soc. Am. A 12 (9) (1995)
18771883.
[23] A. Rosenfeld, C.-Y. Wang, A.Y. Wu, Multispectral texture,
IEEE Trans. Syst. Man and Cybernet. SMC-12 (1) (1982)
7984.
[24] T.M. Lehmann, C. Palm, Local line search for illuminant
estimation in real world color scenes, J. Opt. Soc. Am. A
18 (11) (2001) 26792691.
[25] B.A. Wandell, Foundations of Vision Science, Sinauer Ass.,
Sunderland, MA, USA, 1995.
[26] T.S.C. Tan, J. Kittler, Colour texture analysis using colour
histogram, IEE Proc. Vision Image Signal Process. 141 (6)
(1994) 403412.
[27] M. Mirmehdi, M. Petrou, Segmentation of color textures,
IEEETrans. Pattern Anal. Mach. Intell. 22 (2) (2000) 142159.
[28] A. McCabe, T. Caelli, G. West, A. Reeves, Theory of
spatiochromatic image encoding and feature extraction, J. Opt.
Soc. Am. A 17 (10) (2000) 17441754.
[29] A. Bradley, P. Jackaway, B. Lovell, Classication in
scale-space: applications to texture analysis, 14th International
Conference on Information Processing in Medical Imaging,
Ile de Berder, France (1995) 375376.
[30] A.N. Kolmogorov, Three approaches to the quantitative
denition of information, Int. J. Comput. Math. 2 (1968)
157168.
[31] Database VisTex of color textures of MIT, USA:
ftp://whitechapel.media.mit.edu/pub/VisTex.
[32] Database of color textured barks of University of Koblenz,
Germany: ftp://ftphost.uni-koblenz.de/outgoing/vision/
Lakmann/BarkTex.
[33] R. Lakmann, Statistische Modellierung von Farbtexturen,
Ph.D. Thesis, University Koblenz-Landau, F olbach, Koblenz,
1998 (in German).
[34] S.W. Zucker, D. Terzopoulos, Finding structure in
Co-occurrence matrices for texture analysis, Comput. Graph.
Image Process. 12 (1980) 286308.
[35] J.F.R. Ilgner, C. Palm, A.G. Sch utz, K. Spitzer, M.
Westhofen, T.M. Lehmann, Colour texture analysis for
quantitative laryngoscopy, Acta Otolaryngol. 123 (2003)
730734.
About the AuthorCHRISTOPH PALM studied Computer Science at the Aachen University of Technology (RWTH), Germany, and
received his Diploma degree in 1997. Between 1997 and 2001 he was Ph.D. student at the Institute of Medical Informatics at the University
Hospital of the RWTH. During his Ph.D. research, his research interests covered all aspects of pattern recognition, texture classication
and color image processing. He is co-editor of a proceedings volume and author of several scientic conference and journal papers. He is
peer reviewer of the Optical Society of America. In 2001 Christoph Palm joined the Aixplain AG, Aachen. Beside image processing, his
research focus is now enlarged to information retrieval and statistical machine translation.