IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO.
3, MARCH 2006
Texture Synthesis: Textons Revisited
Dimitrios Charalampidis, Member, IEEE
Abstract—This paper introduces a technique for synthesizing
natural textures, with emphasis on quasiperiodic and structural
textures. Textures are assumed to be composed of three com-
ponents, namely illumination, structure, and stochastic. The
contribution of this work is that, in contrast to previous tech-
niques, it proposes a joint approach for handling the texture’s
global illumination, irregular structure, and stochastic component
which may be correlated to the other two components. Further-
more, the proposed technique does not produce verbatim copies
in the synthesized texture. More specifically, a top-down approach
is used for extraction of texture elements (textons) in which, in
contrast to previous texton-based approaches, no assumptions
regarding perfect periodicity are made. The structure itself can
be modeled as a stochastic process. Consequently, textons are
allowed to have irregular and nonidentical shapes. In the synthesis
stage, a new nonregular textural structure is designed from the
original one that defines the place holders for textons. We call
such place holders empty textons (e-textons). The e-textons are
filled in by a representative texton. Since e-textons do not have
identical shapes, a texton shape-matching procedure is required.
After adding the illumination to the structural component, a
strictly localized version of a block sampling technique is applied
to add the stochastic component. The block sampling technique
combined with the addition of the illumination component pro-
vides a significant improvement in the appearance of synthesized
textures. Results show that the proposed method is successful in
synthesizing structural textures visually indistinguishable to the
original. Moreover, the method is successful in synthesizing a
variety of stochastic textures.
Index Terms—Periodicity detection, textons, texture synthesis.
I. INTRODUCTION
T EXTURE classification, segmentation [1], [3], and syn-
thesis [4]–[23], [26]–[30] find important applications in
computer and machine vision. In particular, texture synthesis
applications include computer graphics, image modeling, cam-
ouflaging, and in general, generation of visually realistic nat-
ural environments. In most cases, information extracted from an
original texture patch is used to synthesize different texture real-
izations, hopefully visually indistinguishable from the original.
Texture synthesis techniques proposed in the literature follow a
structural, statistical, or hybrid approach. Human perception is
sensitive to structure and periodicity. Thus, the structural or pe-
riodic components of a texture must be given special attention.
Parametric modeling is one of the popular approaches and
includes predictive techniques such as autoregressive (AR)
models, and moving average (MA) models. Chellappa et al.
Manuscript received July 11, 2004; revised February 12, 2005. The associate
editor coordinating the review of this manuscript and approving it for publica-
tion was Dr. Attila Kuba.
The author is with the Department of Electrical Engineering, College of En-
gineering, University of New Orleans, New Orleans, LA 70148 USA (e-mail:
dcharala@uno.edu).
Digital Object Identifier 10.1109/TIP.2005.860604
1057-7149/$20.00 © 2006 IEEE
777
[4] proposed a two-dimensional (2-D) noncausal AR model to
characterize the gray level at a pixel as a linear combination of
gray levels at nearby locations in all directions assuming the
existence of an additive white noise variable. Eom [6] proposed
a class of random field models, called generalized circular
AR (GCAR) models for isotropic and anisotropic textures.
The GCAR models have noncausal neighbors and circular or
elliptical correlation structure. AR parameter values are equal
if they are on the same circle or ellipse. Eom [5] also proposed
a random field model based on MA time-series model. A 2-D
blind system identification scheme for texture synthesis was
proposed by Chi et al. [7]. The technique was based on 2-D
linear shift-invariant systems using a 2-D linear prediction
error filter. These techniques may be effective in producing
visually realistic random textures, but they do not successfully
model structural or directional textures. In contrast to the above
techniques, the proposed work deals explicitly with structure
detection, producing textures visually similar to the original.
Multiresolution approaches are common in texture synthesis.
Zhang et al. [8] proposed a multiresolution statistical model,
consisting of random fields in wavelet sub-bands. A complex
random field model containing long-range and nonlinear spatial
correlations is constructed from several simpler ones. Paget
et al. [9] proposed a noncausal, nonparametric, multiscale,
Markov random field model incorporating local annealing to
obtain larger realizations of texture. Sivakumar and Goutsias
[10] used constrained Gibbs random fields to model images by
their size density, incorporating multiresolution information
into image modeling. Jacovitti et al. [11] proposed a twin-stage
texture synthesis-by-analysis method to approximate first and
second order texture distributions. The binary behavior of a
given prototype is represented by means of a hard-limited
Gaussian process, and a texture is synthesized by passing the
binary process through a linear filter followed by a histogram
equalizer. This work was extended by Campisi et al. [12] by
operating in both spatial and multiresolution domains using cir-
cular harmonic functions. Campisi et al. [13] also applied this
basic concept to texture coding. Since structure may be a signif-
icant component only for particular resolutions, multiresolution
approaches may be able to generate both random and structured
textures; however, they deal with the structural characteristics
only indirectly. On the other hand, the proposed technique
explicitly searches for the structural component, which makes
it more suitable for synthesizing structural textures.
Another category of texture synthesis techniques is based on
block sampling. Yu et al. [14] proposed a multiresolution block
sampling technique. The texture was split into blocks, which
were randomly rearranged by incorporating a directional con-
straint. Lam et al. [15] presented a technique in which texture
blocks are sampled from the original texture and placed in a
778
raster order by examining the joint occurrence of texture dis-
crimination features and neighborhood pixel similarity. Parada
et al. [16] synthesized textures using image pyramids and self-
organizing maps, based on an extension of the Efros and Leung
technique [17]. Each pixel centered in a block of a randomly
initialized patch was substituted by the central pixel of one of
the most similar blocks in the original image to preserve local
similarity. A similar approach has been followed by Wei and
Levoy [18] who sped up the process using vector quantization.
De Bonet [19] introduced a technique in which the input texture
was first analyzed by measuring the joint occurrence of multiple
resolution discrimination features. Then, a texture was synthe-
sized by sampling successive spatial frequency bands from the
input texture, conditioned on the joint occurrence of features
at lower spatial frequencies. Efros and Freeman [34] proposed
an image quilting technique for texture synthesis. Sample patch
regions from the original image are stitched together along op-
timal seams to generate a new output. This technique has been
extended by Kwatra et al. [35] and Nealen and Alexa [36].
In [35], the size of the patch is not chosen a-priori. Instead, a
graph-cut technique is used to determine the optimal patch re-
gion for any given offset between the input and output textures.
In [36], variable patch sizes are used for overlapping and re-syn-
thesizing regions between adjacent patches. An interesting work
has also been presented in [37] by Dischler et al.. Elementary, ir-
regular shaped textural elements (particles), and their spatial ar-
rangement are detected in a texture patch, and a similar texture is
constructed. In general, block sampling techniques may also in-
directly retain the local image structure and directionality. Nev-
ertheless, the synthesis phase may become unstable resulting in
visual artifacts, or tend to reproduce verbatim copies of the orig-
inal texture. Although the proposed technique includes a block
sampling process for adding the stochastic component to the
texture, this process is stable since it operates on the already gen-
erated structure and illumination components. Furthermore, the
proposed technique does not lead to verbatim copies due to the
addition of the global illumination and stochastic components,
and the due to the synthesized image’s irregular structure.
Some other interesting works include the one by Popat and
Picard [20], which used a cluster-based model to summarize
pixel transition probability mass functions. Zhu et al. [21] pro-
posed a Markov chain Monte Carlo algorithm to sample Julesz
ensembles. The technique finds the statistics defining the en-
semble given an image set. Based on an extension of the tradi-
tional single site Gibbs sampler, it generates random textures by
moving along the directions of filter coefficients. Another tech-
nique based on the Gibbs sampler and stochastic approxima-
tion was proposed in [38] by Gimel’farb. The model takes into
account gray-level pixel interactions, and uses an iterative ap-
proach to recover the strength of those interactions. These tech-
niques also do not handle structure directly.
Two of the early structural approaches to texture were by Vil-
nrotter et al. [22], which belongs to the class of bottom-up struc-
tural techniques, and Matsuyama et al. [23], which is a top-
down structural technique. Bottom-up techniques extract tex-
ture elements and then describe their attributes and spatial re-
lations, while top-down techniques find the structure character-
istics and then identify the basic texture elements. An opera-
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 3, MARCH 2006
tional definition of textural elements is presented in [33]. Al-
though the second approach seems to be more effective, both
techniques proposed in [22], [23] yield visually artificial results.
A top-down approach [24] and a feature based approach [25]
were also proposed for periodicity extraction. Liu et al. [31] pro-
posed a method to detect periodicity from frieze and wallpaper
groups, including hexagonal structures. This work was extended
in [32] for synthesis of near-regular textures, while deformable
textures were also considered. Gimel’farb [39] has also pro-
posed a modification of his Gibbs-based technique [38] for pe-
riodic textures. Another class of texture synthesis methods that
can be viewed as an extension of the early structural approaches
is based on texture decomposition in deterministic and inde-
terministic components. Campisi et al. [26] assumed that tex-
tures have a distribution significantly different than Gaussian to
show that the indeterministic component can be predicted from
the deterministic one by means of suitable nonlinear schemes.
Francos et al. [27] orthogonally decomposed the deterministic
component into a harmonic and a generalized-evanescent com-
ponent. A two-stage extension of this algorithm was proposed
by Francos et al. [28]. The first stage estimated the number of
harmonic and evanescent components, and a suboptimal ini-
tial estimate for the parameters of their spectral supports. The
second stage refined these estimates by iterative maximization
of the observation likelihood function. Hsu and Wilson [29] pro-
posed a two component model for texture analysis and synthesis
that was based on texture block affine coordinate transforma-
tions, and a stochastic residual component. One of the main dif-
ferences between [29] and the proposed technique, is that [29]
does not identify the global basic structure or the global illumi-
nation components. The affine transformations result in a tex-
ture whose structure is essentially similar to the original. The
proposed technique produces a different structure, based on the
properties of the original. Furthermore, in contrast to [29], the
correlation between the stochastic component and the structure
and illumination components is considered.
This paper introduces an approach for synthesizing natural
textures, with emphasis on structural textures. Textures are
assumed to consist of three components, namely illumination,
structure, and stochastic. The process with which the structural
component is obtained is based on a top-down approach for
extraction of texture elements (textons). However, no assump-
tion is made regarding perfect periodicity, and the structure
is modeled as a stochastic process, parametrized by the mag-
nitudes of the frequency components around the fundamental
frequencies. Consequently, textons may have irregular and non-
identical shapes. In the synthesis stage, a new textural structure
is designed as a nonregular mesh that defines the place holders
for textons called empty textons (e-textons). The e-textons
are filled in by a representative texton. Since e-textons do not
have identical shapes, a texton shape-matching procedure is
required. After adding the illumination component to the struc-
tural component, a strictly localized version of a block sampling
technique incorporates the stochastic component. The combi-
nation of the block sampling technique with the addition of the
illumination component provides a significant improvement in
the appearance of synthesized textures. The contributions of
this work are the following. The technique deals explicitly with
CHARALAMPIDIS: TEXTURE SYNTHESIS: TEXTONS REVISITED 779

periodicity detection on irregular structures, and the identifi-

cation of irregular-shaped textons. Furthermore, it provides a
global treatment of the illumination and structural variations,
and introduces a technique for adding a stochastic component
which is correlated to the other two components. Although the
method was designed for synthesizing structural textures, a
variety of stochastic textures is also effectively synthesized.
This paper is organized as follows. Section II introduces the
proposed technique, and Section III presents results that illus-
trate the ability of the algorithm to generate realistic natural and
especially structural textures. Finally, Section IV closes with
some concluding remarks.

II. PROPOSED METHOD

This section presents the proposed technique consisting of
two stages, namely analysis and synthesis, presented in Sec-
tions II-A and II-B, respectively.

A. Analysis Stage
The analysis stage deals mainly with the identification of the
quasiperiodicity underlying the textural structure, and the iden-
tification of textons. Periodicity can be determined from the
image autocorrelation, either in the spatial or in the frequency
domain. The frequency domain approach searches for peaks that
identify the fundamental frequencies in the image. The spatial
domain approach searches for peaks that correspond to the cor-
relation shifts that indicate high similarity between the original
and shifted image versions. In this work, the frequency domain
autocorrelation is used, because the proposed technique requires
determining the fundamental frequencies. The frequency do-
main autocorrelation of an image of size is defined as
(1)
Fig. 1. Example of texton and structure determination. (a) Original patch and
where and are the frequency and (b) its frequency spectrum. Frequency response of (c) first and (d) second Gabor
pixel location vectors, matrix is an all-ones matrix, and filters. Spatial domain frequency components of (e) the first and (f) the second
is the Kronecker product. The Kronecker product is used to fundamental frequencies and (g) combined component. Spatial domain ridges
of (h) the first and (i) second fundamental frequency. (j) Composite mesh of
provide higher resolution in . The autocorrelation center textural structure. Texton labeling (k) with mesh, (l) without mesh, and (m) final.
is located at . Periodicity in images may exist in
various orientations, but two orientations are generally suffi- function maximized at . The frequency pair chosen is the one
cient for determining the basic structure. Higher peaks in that maximizes the measure
are a stronger evidence of periodicity. Peaks that correspond to (3)
fundamental frequency harmonics are aligned on a line passing
through the autocorrelation center , and the fundamental The relation between fundamental frequency and corre-
frequency is generally defined by the peak closest to the . sponding displacement vector is expressed as
Therefore, it may be desirable to choose peaks whose frequency
(4)
location vectors define an angle significantly different than 0 .
For the reasons mentioned above, a measure that depends on The frequency bands located around the fundamental fre-
the magnitude of two autocorrelation peaks, the angular dis- quencies hold the texture’s basic structure information. Two
tance of their frequency location vectors, and their distance from Gabor filters , with frequency responses centered
is used to choose the pair of peaks. Considering peaks at the two fundamental frequencies and are used to iso-
in located at , with magnitudes , late the two bands. An example is shown in Fig. 1. The orig-
the measure for peaks and is defined as inal image is presented in Fig. 1(a), and its frequency spec-
(2) trum in Fig. 1(b). The frequency responses of the two Gabor
filters and are shown in Fig. 1(c) and (d), re-
where is a measure of the angle between the two spectively. Fig. 1(b)–(d) confirm that the Gabor filter central
peak locations that is minimum for peaks existing in the same frequencies and indeed correspond to frequency do-
orientation. Moreover, is a circularly symmetric distance main peaks. The spatial domain filtering results corresponding
780 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 3, MARCH 2006

1) Structural Component: The structural component is

designed based on information extracted from the original
texture’s structure. As mentioned before, the frequency bands
around the fundamental frequencies hold the texture’s basic
Fig. 2. Textons extracted from second texture of Fig. 1 in the analysis stage. structure information, which can be expressed as

to and are shown in Fig. 1(e) and (f), and the com-
bined result in Fig. 1(g) (negative pixel values are not visible).
Fig. 1(e) and (f) illustrate two approximately sinusoidal surfaces (6)
with slowly varying frequencies and orientations. Determining
local maxima allows forming ridges that correspond to the sinu- In (6), , for , where
soidal positive peaks as it is shown in Fig. 1(h) and (i). If these is the image size. Moreover, and ,
two images are combined, a nonregular mesh representing the are the magnitudes and phases of the , -th frequency com-
basic textural structure is obtained. This is shown in Fig. 1(j). ponents around in a neighborhood specified by . Using
This mesh appears to be consisting of place holders that, if filled trigonometric identities, (6) can be written as
in with appropriate textons, the original texture would be syn-
thesized. In this work, these place holders are called e-textons
(empty textons). (7)
Next, the e-texton locations and shapes are determined. The
process is initialized by finding a zero-valued pixel in the mesh.
This pixel which is part of an e-texton, say , is replaced by for
label , and region growing is performed until all ’s pixels
are replaced by label . The process is repeated until all e-tex- and
tons are filled in with a label. The result of this process is shown
in Fig. 1(k) for the example of Fig. 1. Then, the mesh is re-
moved, as shown in Fig. 1(l), and its pixels are assigned to Considering that the fundamental frequency magnitudes
the neighboring e-textons using a mode filter. The filter substi- are larger than , , functions are slowly
tutes the pixel value in the center of a sliding window by the varying about . In general, the structure mesh is not signif-
most frequent pixel value in the window. The result is shown in icantly affected by the slow variations of . However, it is
Fig. 1(m). Finally, since each label corresponds to a texton possible that these variations may cause a problem to the local
, the image can be easily split into several textons as it is maxima detection procedure. More specifically, it is possible
shown in Fig. 2. It is clear from the examples presented in Figs. 1 that higher ridges “overshadow” nearby lower ridges, resulting
and 2, that irregular shaped textons can be determined. In case in a discontinuous or broken mesh. In order to assist the local
a texton contains at least one pixel on the image border, it is maxima detection procedure, all ridges should have the same
deleted from the collection of textons as being incomplete. The height. The solution to this problem is to set the func-
number of textons obtained in the analysis stage is denoted as tions equal to a constant. In this case, is represented as a
. fixed-magnitude sinusoidal carrier, which is phase modulated by
The textural structure can be described by the magnitudes of . Functions represent the slowly varying phases and
the most significant spatial frequency components around directions of the two structural components corresponding to .
the fundamental frequencies and . The illumination com- Thus, the local maxima of can be accurately detected.
ponent can be described by the magnitude of the most The gradient of holds information about the phase and
significant low frequency components around . The con- direction changes of the structure. Obtaining the gradient of
struction of a new structure, the handling of the illumination with respect to yields
component, and the choice of and will be described
in more detail in the synthesis stage.
(8)
B. Synthesis Stage
As mentioned earlier, a texture is synthesized considering where .
a structural, an illumination, and a stochastic component. The Then
basic texture model is

(5)

where and represent the structural and illumi- (9)

nation components, and is an operator that introduces
the stochastic component to in order to pro- where , and . In
duce the final synthesized texture . Next, the several the approximation of (9), it was assumed that is signifi-
synthesis steps are presented. cantly larger than . Therefore, only the
CHARALAMPIDIS: TEXTURE SYNTHESIS: TEXTONS REVISITED
Fig. 3. Constructed textural structure corresponding to texture in Fig. 1.
term for which is significant for
the denominator, and the terms for which are
significant for the numerator. In order to design a new structure
similar to the original and at the same time introduce sufficient
randomness, the phase gradient in (9) should have similar prop-
erties in both the original and new structures. For instance, the
or equivalently the terms do not depend on and
should be identical. Additional constraints could be imposed on
the -dependent part of the gradient, namely ; how-
ever, no constraint is used here. Thus, structure in (6) is
treated as a random process parametrized by magnitudes .
More specifically, the phases are considered to be uni-
form random variables in the interval , and the
larger magnitudes in each of the two bands specified by the two
Gabor filters and are used in (6) to produce the
new structure. The procedure described here provides sufficient
randomness to the structure, while preserving its main charac-
teristics. The choice of is not crucial in the generation
of the new structure. For instance, a highly regular structure,
such as a grid, has just two significant frequency components
in the neighborhood around fundamental frequencies and
, i.e., the ones at and . Introducing a random phase to
several components around and will simply cause a spa-
tial shift to the entire structure, since the frequency components
magnitudes other than those of and are insignificant;
thus, . On the other hand, an example of a less
regular structure, such as the one corresponding to the texture
of Fig. 1, is shown in Fig. 3. It can be observed that although
the structures of Figs. 1(j) and 3 possess similar characteristics,
they are not identical.
The size of the synthesized image depends on the range of
; thus, an arbitrarily large texture can be synthesized. How-
ever, the structure as defined in (6) is periodic with periods
and . Therefore, the structure synthesis approach must be
modified so that slowly varying random phases are used
in the new structure to avoid a periodic repetition.
2) Texton Processing: The same procedure as in the anal-
ysis stage is applied to to obtain the mesh, and to iden-
tify the e-textons included in it. Next, the new structure’s e-tex-
tons are replaced by a representative texton. However, textons
(or e-textons) appear in different shapes and sizes. For that pur-
pose, a texton-matching technique is required. The operation
that matches texton to texton is denoted as
. An example of texton-matching is shown in Fig. 4. The
texton shape in (c) matches that of (a), while it still retains the
characteristics of the texton in (b). The matching approach used
in this paper is not content-based but shape-based. Therefore, a
781
Fig. 4. Matching example. Texton in (b) has been matched to texton in (a) and
shown in (c).
Fig. 5. (a) Representative texton T and (b) synthesized texture using only the
structural component.
texton and an e-texton can be matched. In order to match two
textons and , two bounding boxes and are deter-
mined for and , respectively. Then, the vertical and hori-
zontal coordinates of both bounding boxes are normalized be-
tween 0 and 1. For instance, the upper left pixel of both and
is assigned the coordinates (0,0), and the lower right pixel the
coordinates (1,1). Then, texton is matched to texton by re-
placing all pixels in by the most similar-coordinate pixels in
.
Although different variations of the technique can be used,
in this work, the matching procedure is used to substitute each
e-texton by the same representative texton, say . Texton is
determined from the following procedure. First, all textons ,
are matched to a single texton, say to obtain
identically shaped textons , .
The choice of is not crucial to the algorithm; however, it is
preferable that ’s size is not significantly different than the
average texton size. Each pixel in texton is calculated as the
median of all corresponding pixels. The median is preferred
over the mean, because it ignores “outlier” textons. Fig. 5(a)
depicts the representative texton for the example of Fig. 1,
and Fig. 5(b) presents the synthesized texture at this stage of
the algorithm, denoted as . Although the same texton
substitutes the e-textons in the new structure, the e-texton shape
irregularity provides a relatively realistic appearance to the new
texture. Nevertheless, this is not the algorithm’s final product.
The synthesized textures are made more realistic by incorpo-
rating the illumination and stochastic components.
3) Illumination Component: Textures synthesized using
structural and texton information provide relatively realistic re-
sults due to structure randomness introducing shape variations
to the representative texton . However, the perceived image
intensity may not be uniform over the entire image. The illumi-
nation component is defined here as a slowly varying
intensity component, with variations significantly smaller than
the basic structure’s fundamental frequencies. In this work, the
782
frequency components around are chosen to
represent the illumination component
(10)
where , for . Moreover,
and are the magnitude and phase of the -th
frequency component located around . Similarly to the
structure generation approach, is a random variable in
the interval , while is identical to the corresponding
magnitude obtained from the original texture. Practically,
has to be slowly varying to avoid a periodic repetition of the
illumination component which will be apparent in synthesized
textures larger than the original. The illumination and structural
components are combined as follows:
(11)
where and are the average and standard devi-
ation obtained from ’s pixel values. Equation (11) sets the
average and standard deviation of the illumination and structural
components product to that of the original texture. Examples of
will be shown in the results section.
4) Stochastic Component: Equation (5) implies that the sto-
chastic component generation (SCG) is not a simple additive
process. The basic idea behind the SCG technique is based on
the approach proposed by Efros and Leung [17], which is a non-
parametric method that grows a new image from an initial seed
sampled from the original image. One pixel is synthesized at a
time assuming a Markov random field model. Instead of using
the conditional distribution of a pixel
located at given its neighborhood of already synthesized
pixels, , the algorithm estimates the pixel directly
from the original image by finding all similar neighborhoods.
More specifically, a pixel in one of the original image’s similar
neighborhoods substitutes the corresponding pixel in the new
image’s neighborhood. The goal of this method is to preserve
local similarity.
The disadvantage of [17] as an independent algorithm is its
reliance on local neighborhoods. Hence, the global texture prop-
erties may not be preserved. Moreover, similarity between two
relatively small neighborhoods does not imply that they repre-
sent different realizations of the same process. In addition, the
sequential nature of the algorithm may lead to a pixel neigh-
borhood highly dissimilar to any neighborhood in the original
image. Thus, the algorithm may become unstable resulting in
synthesis artifacts. Instability depends on the choice of the local
neighborhood size. It is expected that a large neighborhoods or
employment of multiresolution analysis as in similar techniques
[16], [19] may provide stability. Nevertheless, similarity mea-
sures from large neighborhoods may not be effective, especially
for highly irregular structures.
A similar approach to the one in [17] is used in this paper,
however, only to generate the stochastic component given the il-
lumination and structural component image defined in
(11). The aim is to substitute a pixel in by a pixel in the
original image given a considerably small neighborhood, and
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 3, MARCH 2006
given the relative pixel position in the corresponding texton. As
a reminder, the representative texton is used to substitute all
e-textons in the new structure based on a texton shape-matching
operator . Let the number of pixels in
be equal to . Each pixel in texton is given a unique in-
teger label taken from the interval . The “texton” re-
sulted by substituting each pixel in by its corresponding label
is denoted as . Texton substitutes all e-textons in the orig-
inal and new structures based on the same matching operator
to produce the label images and
.
Since the images are discrete, the matching procedure may
result in loss or duplication of some of the labels in . Never-
theless, this does not cause a problem because pixel duplication
only affects adjacent pixels with similar neighborhoods. Fur-
thermore, in the case that labels are lost for some textons, there
is generally a sufficient number of textons in which this label
is preserved. Considering an image , the conditional dis-
tribution for the pixel located at given its neighboring pixels,
, and given its corresponding label
is .
The stochastic component is generated as follows. The value
of the central pixel in a 4 4 neighborhood in is sub-
stituted by the value of the pixel corresponding to the most
similar 4 4 neighborhood in the original image subject to the
constraint that pixel ’s corresponding label is equal
to . This approach has some significant advantages.
First, it is considerably fast since only few neighborhoods in the
original image will be compared to the neighborhood under con-
sideration in the new image. In case that there are textons
in the original image, there will be approximately neighbor-
hoods for a particular label (not exactly due to loss or du-
plication of labels). This number is relatively small, depending
on the original image and the texton sizes. Furthermore, SCG
works locally in order to provide sufficient randomness. At the
same time, it is based on the existing component, so
that there is no stability problem. Finally, the constraint imposed
by the label images and allows transferring
of information only between neighborhoods in the original and
new images that are actually relevant since they represent sim-
ilar texton locations.
In order to provide a more random initial condition to the
SCG process, random noise may be added to . Similarly
to the approach where ’s pixels are calculated as the median
of all corresponding pixels of textons , a deviation texton
is calculated as the standard deviation of all corresponding
pixels of textons . Then, the matching procedure is used to
substitute all e-textons in the new structure by to assign a
standard deviation to each pixel in the new texture. Assuming
that the median approximates the mean, each pixel’s distribu-
tion is modeled as a Gaussian with known mean and standard
deviation. These distributions generate additive noise for each
pixel in assuming pixel independence to provide a good
initialization to the SCG.
At certain steps in the synthesis algorithm, it may be benefi-
cial to remove artifacts generated from previous synthesis steps.
More specifically, the ratio of a smoothed version of the orig-
inal image’s frequency magnitude over a smoothed version of
CHARALAMPIDIS: TEXTURE SYNTHESIS: TEXTONS REVISITED
Fig. 6. Block diagram of the proposed texture synthesis approach.
the synthesized image’s frequency magnitude is multiplied to
the synthesized image’s Fourier Transform. Thus, some artifacts
such as frequency components nonexistent in the original image
are removed. This step is called spectrum adjustment procedure
(SAP).
5) Effect of Illumination on the Stochastic Component: The
slowly varying illumination component modulates the structural
component assigning a similar average intensity to nearby tex-
tons, in a similar manner as in the original image. Based on the
SCG technique, pixel values in low (high) intensity regions in
the new image tend to be substituted by pixel values from low
(high) intensity regions in the original texture. This is desirable,
since the illumination and stochastic components may be corre-
lated in the original texture, so that the textural properties may
be slightly varying with intensity. Fig. 6 summarizes the anal-
ysis and synthesis stages of the proposed algorithm.
III. RESULTS
In this section, the proposed method is tested on a number
of structural and stochastic textures. The textures used in the
experiments are obtained from the Brodatz album as in most
of the works in the literature. The original patches are of size
64 64. The weighting function used for the measure in
(2) is a Gaussian function with standard deviation . fromthe texture sampling category. Furthermore, a variation of
Some texture synthesis examples of the proposed method are
presented in Figs. 7–9. Furthermore, comparisons with three
other existing techniques are presented in Fig. 7. The descrip-
tion of the first five image columns in these figures starting from
left to right is as follows: original textures, generated structure
meshes, structural components, structure and illumination com-
ponents, final synthesis results. The time required to synthesize
a new texture patch is almost proportional to the number of tex-
tons in the synthesized image. An average of about 0.4 s/texton
is needed on a P4, 2.8-GHz, 512-Mb computer using a nonop-
timized MATLAB code.
783
Fig. 7 presents synthesis results for structural textures. Some
generated textural structures are more periodic than others, fol-
lowing the properties of the original texture patch. For instance,
highly periodic textures are the ones in (c), (f), and (j). Tex-
tures (a), (e), (g), (h), and (k) are less periodic, while the ones
in (b), (d), (i) are even more irregular. An important observa-
tion is that the purely structural components shown in the third
column do not appear to be completely artificial, although the
same texton was used to fill in the e-textons. As a reminder,
this is due to the nonuniformity of the e-texton shapes. In par-
ticular, textures (c), (e), (f), (h), and (i) seem to require no sig-
nificant stochastic or illumination component. Therefore, the re-
sults shown for those textures in the third column could be the
final synthesis products. This would be useful in compression
applications, since the only information required from the orig-
inal texture would be the few magnitude components around the
fundamental frequencies , and the representative texton .
The addition of the illumination component provides a more re-
alistic appearance to textures. It requires eight frequency com-
ponent magnitudes around , which is not a significant
overhead for compression applications. The improvement pro-
vided by the illumination component is enhanced significantly
by the incorporation of the stochastic component. For textures
with significant illumination component such as (c) and (k), the
SCG procedure is allowed to occasionally sample from high in-
tensity and sometimes from low intensity regions in the orig-
inal image. This results in variations of the textural characteris-
tics following the illumination variations, for cases where some
correlation exists between intensity and texture in the original
image. Maintaining the possible correlation between intensity
and texture, while achieving textural continuity, is one of the im-
portant properties of the proposed technique. It should be noted,
however, that the successfulness of the results is not simply due
to the block sampling-based SCG procedure. For instance, the
visual correspondence between column four images (prior to
SCG) and column five images (after SCG) is obvious. The infor-
mation provided by previous algorithmic steps allows the small
neighborhoods used in SCG to capture local information while
maintaining stability.
Fig. 7 also presents comparisons with three other texture
synthesis techniques. Visual comparisons are important in
order to illustrate how the proposed technique is “better.”
More specifically, comparisons with three other works (Efros
[17], Liu [31], [32], Gimel’farb [39]) are presented. These
techniques are chosen as being representative of their category.
The Efros technique is chosen since it is a representative work
this technique has been used as a component in the proposed
technique. The Liu technique is one of the latest advanced
techniques, and it is a representative periodicity detection/tiling
technique. Periodicity is detected in a given texture patch in the
form of texton translational symmetry vectors. These vectors
identify a lattice, and consecutively, the basic texture tiles
(called minimum tiles). Then, rectangular windows circum-
scribing each minimum tile (called maximum tiles) centered on
the lattice points and on the half-way shifted lattice points can
be obtained. In the synthesis phase, the maximum tiles obtained
from the original patch are placed on the lattice one by one,
784 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 3, MARCH 2006

Fig. 7.(a)–(f) Synthesis results for structural textures. Columns from left to right: original textures, structure meshes, structural components, structural with
illumination components, final synthesis results of the proposed method, synthesis using Gimel’farbs’ method, and synthesis using Liu’s method. Synthesis results
for structural textures. Columns from left to right: original textures, structure meshes, structural components, structural with illumination components, final synthesis
results of the proposed method, synthesis using Gimel’farbs’ method, and synthesis using Liu’s method.

in an overlapping fashion. The choice of the maximum tile at
every step is based on its similarity with the already placed tiles
at the overlapping region. Small tile shifts are allowed to obtain
a better match. A tile is stitched to the existing ones using
dynamic programming and blending. The basic disadvantage
of this technique with respect to the proposed approach is that
it does not deal with the global illumination and structure and
their relation to the stochastic component. Therefore, tiles or
parts of tiles may appear multiple times in the same image,
which can also be observed in Fig. 7. The Gimel’farb technique
is Gibbs-based. However, it has been modified to be applicable
for the synthesis of periodic textures. The Efros technique
is successful for cases (g) and (k), while it is unstable for
case (i), and creates artifacts for cases (h) and (j). The Liu
and Gimel’farb techniques are both successful in detecting
periodicity and in tiling representative texture elements, for
all examples presented. However, the Gimel’farb technique
produces verbatim copies of the same textural elements, which
may be only acceptable for some regular textures such as (a),
(c), (e), and (f). The Liu technique generates very good visual
results. Nevertheless, one can easily observe that several exact
copies of the same pattern may repeat several times in the
same image. This is easier to detect in cases such as (b) and
(d), while it may not be easily noticeable in other cases. It
should be mentioned that none of the three techniques deals
with the global illumination component or the relation between
the global structure/illumination and the stochastic component
in the texture. A first look at the results of Fig. 7 may suggest
that the three other techniques often produce excellent visual
results. However, this is due to the high structural regularity
CHARALAMPIDIS: TEXTURE SYNTHESIS: TEXTONS REVISITED 785

Fig. 7. (Continued) (g)–(k) Synthesis results for structural textures. Columns from left to right: original textures, structure meshes, structural components,
structural with illumination components, final synthesis results of the proposed method, synthesis using Gimel’farbs’ method, and synthesis using Liu’s method.
Synthesis results for structural textures. Columns from left to right: original textures, structure meshes, structural components, structural with illumination
components, final synthesis results of the proposed method, synthesis using Gimel’farbs’ method, and synthesis using Liu’s method.

caused by tile repetition. Nevertheless, these techniques do
not introduce the “randomness” required for natural texture
appearance, such as in cases (b) and (k). Randomness and
irregularity are desired properties of the proposed technique,
which synthesizes “imperfect” textures similar to the original.
Fig. 8 presents synthesis results for some stochastic textures.
The structure meshes shown in the second column indicate that
although the textures are not structural, there is some basic scale
dependence present. The structural components successfully ex-
tract this information from the original textures. Nevertheless,
this component alone provides an undesired, unnatural periodic
appearance to stochastic textures. Addition of the illumination
component significantly improves the results as it is shown in
all three examples of Fig. 8. Finally, the SCG procedure pro-
vides the required randomness. The effect of the SCG procedure
on the final synthesis results can be examined from a compar-
ison between the forth column and fifth column images. The
similarity between these images is apparent. The SCG proce-
dure simply “randomizes” the results obtained from the previous
steps. Moreover, the 4 4 neighborhoods used in SCG are con-
siderably small and would be unable to provide a natural visual
appearance by themselves. However, they are small enough to
provide sufficient randomness. The comparisons presented in
this work are concentrated on structural textures. The results
presented in Fig. 8 simply illustrate that although the technique
was developed for structural textures, it is still possible to syn-
thesize some random textures. In general, this is not possible for
most structural texture synthesis techniques.
Fig. 9 shows results for two of the Fig. 7 textures, in which
only two components around the fundamental frequencies were
used for generation of the structure mesh. The final synthesis re-
sults are realistic. However, the artificial periodicity in the tex-
tures is apparent. Therefore, it is preferable to use a relatively
large number of components around . As mentioned earlier,
consideration of relatively insignificant components around the
fundamentals will not have a negative impact in determining the
basic structure.
IV. CONCLUSIONS AND DISCUSSION
This paper introduces a new texture synthesis technique based
on generation and combination of three components, namely the
786
Fig. 8. Synthesis results for random textures. Columns from left to right: original textures, structure meshes, structural components, structural with illumination
components, final synthesis results of the proposed method, synthesis using the Efros and Leung method, and synthesis using Liu’s method.
Fig. 9. Results assuming almost periodic structures. Columns from left to
right: original textures, structure meshes, and final synthesis results.
illumination, structural, and stochastic components. The basic
concept followed in this technique goes back to the idea of iden-
tifying basic textural elements called textons. However, the tech-
nique does not rely on exact periodicity assumptions or on the
regularity of the texton shapes. Therefore, irregular-shaped tex-
tons can be identified and processed. Moreover, this approach
provides the ability to extract “structure” from stochastic tex-
tures, in the sense that some basic textural scale information
can be extracted. Due to the shape irregularity of textons, the
proposed method does not produce blockage artifacts, which
is a significant problem for block sampling techniques, espe-
cially for structural textures. Furthermore, the proposed tech-
nique generates sufficient randomness, regardless of the fact that
a single representative texton is used.
Similar to the work proposed in [31] and [32], the proposed
algorithm can accept some user intervention for choosing
the frequency components required for periodicity detection,
including the number of frequencies around the funda-
mentals. As mentioned in Section II-B1, directly affects
the textural structure’s regularity and periodicity. Moreover,
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 3, MARCH 2006
the phase of the basic frequency components can be chosen so
that the boundaries between adjacent textons are as smooth as
possible in order to avoid possible artifacts caused by texton
stitching. Additionally, investigation of using multiple frequen-
cies including fundamental frequencies in multiple directions,
and also harmonics of the fundamentals may be proven to be
beneficial. This is especially true for textures with hexagonal
structure, but also for textures with complex structured tex-
tons, such as textons with strong symmetry and self-similarity
characteristics.
Future work includes investigation of a more theoretical basis
to obtain the relation between the three textural components,
and to model the structure’s slowly varying direction and phase.
Finally, the basic autocorrelation technique can be expanded to
feature-based autocorrelation or feature matching, which may
be more appropriate for stochastic textures.
REFERENCES
[1] D. Charalampidis and T. Kasparis, “Wavelet—Based rotational invariant
segmentation and classification of textural images using directional
roughness features,” IEEE Trans. Image Process., vol. 11, no. 8, pp.
825–836, Aug. 2002.
[2] D. Charalampidis, T. Kasparis, and L. Jones, “Multifractal and in-
tensity measures for the removal of nonprecipitation echoes from
weather radars,” IEEE Trans. Geosci. Remote Sens., vol. 40, no. 3, pp.
1121–1131, May 2002.
[3] T. Kasparis, D. Charalampidis, J. Rolland, and M. Georgiopoulos, “Seg-
mentation of textured images using fractals and image filtering,” Pattern
Recognit., vol. 34, no. 10, pp. 1963–1973, Oct. 2001.
[4] R. Chellappa and R. Kashyap, “Texture synthesis using 2-D noncausal
autoregressive models,” IEEE Trans. Acoust., Speech, Signal Process.,
vol. 33, no. ASSP-1, pp. 194–203, Feb. 1985.
[5] K. B. Eom, “2-D moving average models for texture synthesis and anal-
ysis,” IEEE Trans. Image Process., vol. 7, pp. 1741–1746, Dec. 1998.
[6] , “Generalized circular autoregressive models for modeling
isotropic and anisotropic textures,” in Proc. Int. Conf. Image Pro-
cessing, vol. 2, Oct. 2001, pp. 129–132.
[7] C.-Y. Chi and C.-H. Chen, “Two-dimensional frequency-domain blind
system identification using higher order statistics with application to tex-
ture synthesis,” IEEE Trans. Signal Process., vol. 49, no. 4, pp. 864–877,
Apr. 2001.
CHARALAMPIDIS: TEXTURE SYNTHESIS: TEXTONS REVISITED
[8] J. Zhang, D. Wang, and Q. N. Tran, “A wavelet-based multiresolution
statistical model for texture,” IEEE Trans. Image Process., vol. 7, no. 11,
pp. 1621–1627, Nov. 1998.
[9] R. Paget and D. Longstaff, “Texture synthesis via a noncausal nonpara-
metric multiscale Markov random field,” IEEE Trans. Image Process.,
vol. 7, no. 6, pp. 925–931, Jun. 1998.
[10] K. Sivakumar and J. Goutsias, “Morphologically constrained GRFS:
Applications to texture synthesis and analysis,” IEEE Trans. Pattern
Anal. Mach. Intell., vol. 21, no. 2, pp. 99–113, Feb. 1999.
[11] G. Jacovitti, A. Neri, and G. Scarano, “Texture synthesis-by-analysis
with hard-limited Gaussian processes,” IEEE Trans. Image Process., vol.
7, no. 11, pp. 1615–1621, Nov. 1998.
[12] P. Campisi and G. Scarano, “A multiresolution approach for texture
synthesis using the circular harmonic functions,” IEEE Trans. Image
Process., vol. 11, no. 1, pp. 37–51, Jan. 2002.
[13] P. Campisi, D. Hatzinakos, and A. Neri, “A perceptually lossless, model-
based, texture compression technique,” IEEE Trans. Image Process., vol.
9, no. 9, pp. 1325–1336, Sep. 2000.
[14] Y. Yu, J. Luo, and C. W. Chen, “Synthesis of directional texture based
on multiresolution block sampling and constrained block movement,” in
Proc. Int. Conf. Image Processing, vol. 2, Oct. 2001, pp. 602–605.
[15] W. L. Lam and B. Zeng, “Fast texture synthesis by feature matching,” in
Proc. Int. Conf. Image Processing, vol. 2, Oct. 2001, pp. 614–617.
[16] P. Parada and J. Ruiz-del-Solar, “Texture synthesis using image pyra-
mids and self-organizing maps,” in Proc. 11th Int. Conf. Image Analysis
and Processing, Sep. 2001, pp. 244–249.
[17] A. Efros and T. Leung, “Texture synthesis by nonparametric sampling,”
in Int. Conf. Computer Vision, vol. 2, Sep. 1999, pp. 1033–1038.
[18] L. Wei and M. Levoy, “Fast texture synthesis using tree-structured vector
quantization,” in Proc. Annu. Conf. Computer Graphics, Jul. 2000, pp.
479–488.
[19] J. S. De Bonet, “Multiresolution sampling procedure for analysis and
synthesis of texture images,” in Proc. Annu. Conf. Computer Graphics,
Aug. 1997, pp. 361–368.
[20] K. Popat and R. W. Picard, “Novel cluster-based probability model for
texture synthesis, classification, and compression,” in Proc. SPIE, Visual
Communication Information Processing, vol. 2094, Boston, MA, Nov.
1993, pp. 756–768.
[21] S. C. Zhu, X. W. Liu, and Y. N. Wu, “Exploring texture ensembles by
efficient Markov chain monte carlo—Toward a trichromacy theory of
texture,” IEEE Trans. Pattern Anal. Machine Intell., vol. 22, no. 6, pp.
554–569, Jun. 2000.
[22] F. Vilnrotter, R. Nevatia, and K. Price, “Structural analysis of natural
textures,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-8, no. 1,
pp. 76–89, Jan. 1986.
[23] T. Matsuyama, S. Miura, and M. Nagao, “Structural analysis of nat-
ural textures by Fourier transformation,” Comput. Vis., Graph. Image
Process., vol. 24, pp. 347–362, 1983.
[24] H. Lin, L. Wang, and S. Yang, “Extracting periodicity of a regular texture
based on autocorrelation functions,” Pattern Recognit. Lett., vol. 18, pp.
433–443, 1997.
[25] V. Starovoikov, S. Jeong, and R. Park, “Texture periodicity detection:
Features, properties, and comparisons,” IEEE Trans. Syst. Man Cybern.
A, Syst. Humans, vol. 28, no. 6, pp. 839–849, Nov. 1998.
787
[26] P. Campisi, A. Neri, and G. Scarano, “Nonlinear prediction in the 2D
wold decomposition for texture modeling,” in Proc. Int. Conf. Image
Processing, vol. 1, 2000, pp. 252–255.
[27] J. Francos, A. Narasimhan, and J. Woods, “Maximum likelihood param-
eter estimation of textures using a wold-decomposition based model,”
IEEE Trans. Image Process., vol. 4, no. 12, pp. 1655–1666, Dec. 1995.
[28] J. Francos, Z. Meiri, and B. Porat, “A unified texture model based on 2-D
wold-like decomposition,” IEEE Trans. Signal Process., vol. 41, no. 8,
pp. 2665–2678, Aug. 1993.
[29] T.-I. Hsu and R. Wilson, “A two-component model of texture for anal-
ysis and synthesis,” IEEE Trans. Image Process., vol. 7, no. 10, pp.
1466–1476, Oct. 1998.
[30] D. Charalampidis, “Texture synthesis based on cluster transition prob-
abilities,” in Proc. SPIE, Visual Information Processing XII, vol. 5108,
Orlando, FL, Apr. 21–24, 2003, pp. 327–338.
[31] Y. Liu, R. T. Collins, and Y. Tsin, “A computational model for periodic
pattern perception based on frieze and wallpaper groups,” IEEE Trans.
Pattern Anal. Mach. Intell., vol. 26, no. 3, pp. 354–371, Mar. 2004.
[32] Y. Liu, Y. Tsin, and W. C. Lin, “The promise and perils of near-regular
texture,” Int. J. Comput. Vis., vol. 62, no. 1-2, Apr. 2005, to be published.
[33] J. Malik, S. Belongie, J. Shi, and T. Leung, “Textons, contours and re-
gions: Cue integration in image segmentation,” in IEEE Int. Conf. Com-
puter Vision, Corfu, Greece, Sep. 1999, pp. 918–925.
[34] A. Efros and W. T. Freeman, “Image quilting for texture synthesis and
transfer,” in Proc. 28th Annu. Conf. SIGGRAPH, 2001, pp. 341–346.
[35] V. Kwatra, A. Schodl, I. Essa, G. Turk, and A. Bobick, “Graphcut tex-
tures: Image and video synthesis using graph cuts,” ACM Trans. Graph.,
vol. 22, no. 3, pp. 277–286, 2003.
[36] A. Nealen and M. Alexa, “Hybrid texture synthesis,” in Proc. Euro-
graph. Symp. Rendering, 2003, pp. 97–105.
[37] J.-M. Dischler, K. Maritaud, B. Levy, and D. Ghazanfarpour, “Texture
particles,” in Eurographics Conf., vol. 21, 2002, pp. 401–410.
[38] G. L. Gimel’farb, “Texture modeling by multiple pairwise pixel inter-
actions,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 18, no. 11, pp.
1110–1114, Nov. 1996.
[39] [Online]. Available: http://www.citr.auckland.ac.nz/ & tilde; georgy/re-
search/ texture/GGvi02.pdf
Dimitrios Charalampidis (S’99–M’01) received the
Diploma degree in electrical engineering and com-
puter technology from the University of Patras, Pa-
tras, Greece, in 1996, and the M.S. and Ph.D. degrees
in electrical engineering from the University of Cen-
tral Florida, Orlando, in 1998 and 2001, respectively.
Since 2001, he has been an Assistant Professor
with the Electrical Engineering Department, Univer-
sity of New Orleans, New Orleans, LA. His research
interests include image and signal processing,
pattern recognition, and neural networks.