10 1 1 3 3907

LLE and Isomap Analysis of Spectra and Colour Images
by
Dejan Kulpinski
B.Sc., Simon Fraser University, 1999
THESIS SUBMITTED IN PARTIAL FULFILLMENT OF
THE REQUIREMENTS FOR THE DEGREE OF
MASTER OF SCIENCE
in the School
of
Computing Science
© Dejan Kulpinski 2002
SIMON FRASER UNIVERSITY
March 2002
All rights reserved. This work may not be
reproduced in whole or in part, by photocopy
or other means, without the permission of the author.

Approval
Name: Dejan Kulpinski

Degree: Master of Science
Title of thesis: LLE and Isomap analysis of Spectra and Colour Images
Examining Committee: Dr. Eugenia Ternovska

Assistant Professor, Computing Science
Chair
_______________________________
Dr. Mark S. Drew, SFU Examiner
Associate Professor, Computing Science
_______________________________
Dr. Brian Funt, Senior Supervisor
Professor, Computing Science
_______________________________
Dr. Torsten Möller, Supervisor
Associate Professor, Computing Science
Date Approved: _______________________________
ii
Abstract
Locally Linear Embedding (LLE) [2] and Isomap [1] techniques can be used to
process and analyze high-dimensional data domains, such as semantics, images, and
colour. These techniques allow creation of low-dimensional embeddings of the original
data that are much easier to visualize and work with then the initial, high-dimensional
data. In particular, the dimensionality of such embeddings is similar to that obtained by
classical techniques used for dimensionality reduction, such as Principal Components
Analysis (PCA).
The goal of this thesis is to show how the above methods can be applied to the
area of colour vision, in particular to images, chromaticity histograms and spectra. Using
the Isomap technique, it was found that the dimensionality of the spectral reflectances
could be as low as 3. For the chromaticity histograms, the 251 original histogram
dimensions were transformed into 5-6 dimensional space. The chromaticity histogram
result is significantly better then the result obtained by PCA on the same data set.
In addition to providing an estimate of the underlying dimensionality of the data,
both the Isomap and LLE techniques were used to produce low-dimensional embeddings
of the high-dimensional data for the purpose of data visualization. These low-dimensional
embeddings were valuable in determining the non-linear relationships that existed among
the members of the original data sets. For example, the relationship among colour
histograms in the embedded space was based on both the chromaticity and image content
while the embedded spectra showed groupings based on the RGB values.
The possible role of the LLE and Isomap in image classification, spectra
reconstruction and RGB-to-Spectra mapping is also examined.
iii
Acknowledgments
I would like to thank Dr. Brian Funt for his guidance and support throughout the
thesis work. He gave me the tremendous amount of feedback, which was invaluable to
completing this thesis.
I would also like to thank Dr. Torsten Möller for his enthusiastic input and
guidance.
I would like to acknowledge the Natural Sciences and Engineering Research
Council, Advanced Systems Institute and Simon Fraser University for their generous
financial support.
iv
Contents
Applications of LLE and Isomap methods ...........................................................................................................1

Approval................................................................................................................................................................. ii
Acknowledgments.................................................................................................................................................iv
Contents .................................................................................................................................................................. v
List of Tables........................................................................................................................................................vii
List of Figures .................................................................................................................................................... viii
Chapter 1.................................................................................................................................................................1
Introduction ............................................................................................................................................................1
Motivation for work...........................................................................................................................................1
Scope of Thesis ..................................................................................................................................................5
Thesis Outline ....................................................................................................................................................7
Chapter 2 ........................................................................................................................................................7
Chapter 3 ........................................................................................................................................................7
Chapter 4 ........................................................................................................................................................7
Chapter 5 ........................................................................................................................................................7
Chapter 6 ........................................................................................................................................................8
Chapter 7 ........................................................................................................................................................8
Chapter 2.................................................................................................................................................................9
Preliminaries...........................................................................................................................................................9
Symbols ........................................................................................................................................................... 10
Spectra......................................................................................................................................................... 11
Chromaticity................................................................................................................................................ 12
Color Constancy.......................................................................................................................................... 13
Principal Component Analysis....................................................................................................................... 14
Variance, Residual Variance and The “Elbow” Technique ..................................................................... 16
Chapter 3.............................................................................................................................................................. 20
Previous Work..................................................................................................................................................... 20
Applications of LLE and Isomap ................................................................................................................... 20
Spectra Dimensionality Estimation ............................................................................................................... 21
Color Histograms and Image Recognition.................................................................................................... 22
Chapter 4.............................................................................................................................................................. 23
The Techniques Explained ............................................................................................................................. 23
The Isomap technique..................................................................................................................................... 23
The LLE Technique ........................................................................................................................................ 25
LLE vs. Isomap.................................................................................................................................................... 27
Chapter 5.............................................................................................................................................................. 28
The application of LLE and Isomap................................................................................................................... 28
Chromaticity Histograms ............................................................................................................................... 28
Input Data .................................................................................................................................................... 28
Determining Dimensionality...................................................................................................................... 31
Embedding................................................................................................................................................... 34
Image Classification ................................................................................................................................... 50
Conclusion................................................................................................................................................... 53
Spectra............................................................................................................................................................. 54
Input Data .................................................................................................................................................... 54
Isomap based embedding............................................................................................................................ 64
LLE based embedding ................................................................................................................................ 65
Spectra Reconstruction and Error Estimation........................................................................................... 67
RGB to Spectra Mapping............................................................................................................................ 71
Conclusion....................................................................................................................................................... 77
Chapter 6.............................................................................................................................................................. 78
v
Discussion............................................................................................................................................................ 78
Chapter 7.............................................................................................................................................................. 80
Appendix A.......................................................................................................................................................... 80
LLE Algorithm................................................................................................................................................ 80
IsoMap Algorithm........................................................................................................................................... 81
LLE Reconstruction Algorithm ..................................................................................................................... 82
References ....................................................................................................................................................... 83
vi
List of Tables
Table 1 - Symbols ............................................................................................................................................... 10

Table 2 - Variance and Residual Variance for Example 1 ............................................................................... 17
Table 3 - Binary Chromaticity Histogram......................................................................................................... 30
Table 4 - Residual Variance for 1050 Chromaticity Histograms based on PCA technique........................... 32
Table 5 - Distortion for 1050 Chromaticity Histograms based on Isomap technique .................................... 33
Table 6 – Distortion for 1050 Chromaticity Histograms based on PCA technique........................................ 33
Table 7 - Residual Variance for 1996 Reflectance Spectra based on PCA technique.................................... 56
Table 8 - Residual Variance for 170 Kodak Reflectance Spectra based on PCA technique......................... 57
Table 9 - Residual Variance for 102 Illuminant Spectra based on Isomap technique .................................... 58
Table 10 - Distortion for 1996 Reflectance Spectra based on PCA technique ............................................... 59
Table 11 – Distortion for 1996 Reflectance Spectra based on Isomap technique ......................................... 59
Table 12 - Distortion for 170 Kodak Reflectance Spectra based on PCA technique ..................................... 60
Table 13 - Distortion for 170 Reflectance Spectra based on Isomap technique ............................................ 60
Table 14 - Distortion for 102 Illuminant Spectra based on PCA technique.................................................... 61
Table 15 - Distortion for 102 Illuminant Spectra based on Isomap technique................................................ 61
vii
List of Figures
Figure 1 – Human face expressions[1]. Dimension X represents the variation in left-right pose of the human
face. Dimension Y represents the variation in up-down pose of the face. The third dimension,
represented y the slider bar, corresponds to the lighting direction in the image .......................................2
Figure 2 – Distortion vs. Dimensionality for PCA (triangles) and Isomap(circles). Arrow indicated the
intrinsic dimensionality as determined by Isomap.......................................................................................3
Figure 3 - Color image and its chromaticity plots ............................................................................................ 12
Figure 4 - PCA Analysis. The red line shows the basis vector for the given data set.................................... 14
Figure 5 – The Elbow Technique [1]. The Red arrow identifies the “elbow” in the graph............................ 18
Figure 6 - Isomap Technique [1]. Swiss role example that demonstrates the transformation and embedding
of the 3 dimensional “Swiss Role” into 2 dimensional space. In image A, for any two points (dark
circles) their Euclidean distance (dashed blue line) may not represent their intrinsic similarity, as
geodesic distance does (solid blue line). Image B shows two points in the Neighborhood graph G
(constructed in Step 1 of Isomap algorithm) and geodesic path (red line) between them (constructed in
step 2 of Isomap algorithm). Image C shows embedding (constructed in step 3 of Isomap algorithm)
which best preserves the shortest path distances in the neighborhood graph. The blue line shows the
straight-line distance between the two points and red line shows the geodesic path. ............................ 23
Figure 7 - LLE Technique. Three steps of the LLE technique. In step 1 of the algorithm K nearest
neighbors are found for each point. In step 2 weights Wij are caclulated. These weights describe the
position of point Xi in relationship to its neighboring point Wj. In step 3 embedding in low
dimensional space is performed using the weights Wij obtained in the step2........................................ 25
Figure 8 - Sample of Image Library used for Chromaticity Histogram experiment. ..................................... 29
Figure 9 - Residual Variance for 1050 Chromaticity Histograms based on PCA technique ......................... 32
Figure 10 - Distortion for 1050 Chromaticity Histograms based on PCA and Isomap techniques. Isomap
graph is labelled with stars, while the the PCA graph is labelled with circles. ...................................... 33
Figure 11 - Each of the 4 panes shows a slice orthogonal to dimension 3 of the subspace embedding with
subspace dimension 1 varying vertically and 2 horizontally. .................................................................. 35
Figure 12 - Similar to Figure 14 except each of the 4 panes shows a slice orthogonal to dimension 2 of the
subspace embedding with subspace dimension 1 varying vertically and 3 horizontally. ...................... 36
Figure 13 - Similar to Figure 15 except each of the 4 panes shows a slice orthogonal to dimension 1 of the
subspace embedding with subspace dimension 2 varying vertically and 3 horizontally. ...................... 37
subspace dimension 2 varying vertically and 3 horizontally ................................................................... 38
viii
Figure 26 - Image Grouping. Indoor vs. Outdoor ............................................................................................. 50
Figure 27 - Image Grouping. Indoor vs. Outdoor ............................................................................................. 51
Figure 28 - Residual Variance for 1996 Reflectance Spectra based on PCA technique ................................ 56
Figure 29 - Residual Variance for 170 Kodak Reflectance Spectra based on PCA technique ...................... 57
Figure 30 - Residual Variance for 102 Illuminant Spectra based on PCA technique..................................... 58
Figure 31 - Distortion for 1996 Spectra based on PCA and Isomap techniques. Isomap graph is labelled
with stars, while the the PCA graph is labelled with circles.................................................................... 59
Figure 32 - Distortion for 170 Kodak Reflectance Spectra based on PCA and Isomap techniques. Isomap
graph is labelled with stars, while the the PCA graph is labelled with circles. ...................................... 60
Figure 33 - Distortion for 102 Illumination Spectra based on PCA and Isomap techniques. Isomap graph is
labelled with stars, while the the PCA graph is labelled with circles...................................................... 61
Figure 34 - 1996 Spectra embedded in 2D space using Isomap ...................................................................... 64
Figure 35 - 170 Spectra embedded in 2D space using Isomap......................................................................... 64
Figure 36 - 1996 Spectra embedded in 2D space using LLE ........................................................................... 65
Figure 37 - 170 Spectra embedded in 2D space using LLE ............................................................................. 65
Figure 38 - Embedding of 170 reflectance Spectra........................................................................................... 71
Figure 39 - Reconstructed RGB values of Spectra embedded in 2D space - low resolution......................... 72
Figure 40 - Reconstructed RGB values of Spectra embedded in 2D space - medium resolution.................. 73
Figure 41 - Reconstructed RGB values of Spectra embedded in 2D space - high resolution........................ 73
ix
Chapter 1
Introduction
Motivation for work
The motivation for using the Isomap and Locally Linear Embedding (LLE)
techniques is based in the fact that the existing techniques for the real dimensionality
estimation, such as Principal Component Analysis (PCA) and its close relative, classic
Multi-Dimensional Scaling (MDS) [22], have problems in terms of overestimating the
true underlying dimensionality of the high dimensional data. Aside from this problem,
both PCA and MSD are not capable of detecting the non-linear structures that are present
in high dimensional data[1].
To demonstrate how the Isomap and LLE can be used for the real dimensionality
estimation, we are going to use the human face expression example from the original
work of Tenenbaum, de Silva, and Langford[1]. In their example, the authors of Isomap
analyze human face expression images in terms of non-linear features they possess. In
other words, they want to learn more about those images than what the perceptual
information encoded in the input data set has to offer. In order to do that, the first thing
they have to do is determine the intrinsic dimensionality of the human face expression
data set. What the intrinsic dimensionality stands for will vary from domain to domain,
but in this particular example, the intrinsic dimensionality of the data set stands for the
number of perceptual factors that combine together to produce an observed image.
Determining the intrinsic dimensionality of a data set allows one to establish the number
of dimensions the embedding of the input set must have in order to adequately represent
1
the perceptual features of the input data. Once the number of intrinsic dimensions is
known and the embedding is performed, the association between the particular dimension
and the non-linear feature it corresponds to can be performed.
Figure 1 – Embedding of human face expressions created by Isomap[1]. Dimension X represents the
variation in left-right pose of the human face. Dimension Y represents the variation in up-down pose
of the face. The third dimension, represented by the slider bar, corresponds to the lighting direction
in the image. [included in the thesis with the written permission of Joshua B. Tenenbaum]
If one were to use PCA and MDS techniques to perform such an analysis on
human face expression images, one would see that PCA and MDS would fail to establish
the 3 dimensional intrinsic dimensionality. Consequently, PCA and MDS would not be
capable of determining the meaning of each of the dimensions in the embedding.
2
However, application of the Isomap to this problem produces much better results.
Firstly, Isomap was able to determine that the intrinsic dimensionality of the face image
data set is 3, as can be seen of Figure 2. Isomap was also able to produce such an
embedding in which each of the three dimensions corresponds to a perceptual variable.
As it can be seen in Figure 1, the X dimension corresponds to the left-right orientation of
the face, the Y dimension corresponds to the up-down orientation of the face and third
dimension, represented by a slider, corresponds to the angle of incident light.
Figure 2 – Distortion vs. Dimensionality for PCA (triangles) and Isomap(circles) of the human face
expression data. Arrow indicates the intrinsic dimensionality as determined by Isomap
Therefore, the conclusion based on the human face expression example by

Tenenbaum, de Silva, and Langford[1] is that Isomap method was capable of detecting
both the underlying intrinsic dimensionality of the high dimensional input data set and
the non-linear features of such data.
Given the above results, we wanted to determine how the Isomap’s ability to
determine the intrinsic dimensionality and non-linear features of the high dimensional
data could be applied to the field of computer vision. In particular, we are interested in
the application of these two techniques in the analysis of chromaticity histograms,
spectral reflectances and color images.
3
The first area this thesis examines is the analysis of the chromaticity histogram
based on a number of unrelated color images. Chromaticity histograms, a derivation of
Color Histograms, are commonly used in computer vision as a tool for image and object
recognition as they lend themselves to very efficient, real-time object recognition
techniques, such as Color Indexing [8]. We wanted to see if Isomap and LLE are capable
of establishing the intrinsic dimensionality and extracting non-linear features from a
library of chromaticity histograms. We were hoping that the Isomap embedding would
identify the relationship between the particular dimensions and perceptual properties of
the histograms not only on the basis of the chromaticity alone, but on the basis of image
content as well. The ability to identify the dimensions that would correspond to the image
content could then lead to a possible use of Isomap for image classification and
recognition.
The other area of application is spectral reflectances. Spectra are typically hard to
work with and difficult to visualize due to their high dimensionality. Typically, spectra
are represented as 31 or 101 dimensional vectors, depending on the range and sampling
rate. The previous research, based on the use of PCA and its derivations [4] [6] [7]
established that the underlying spectra dimensionality is much lower then the 31 or 101
dimensions. Depending on the method and data sets used, the spectra’s underlying
dimensionality was estimated to be between 3 and 8. We wanted to see how LLE and
Isomap would fare in the domain of spectra reflectances compared to PCA based
analysis. We were hoping that Isomap and LLE would able to establish lower intrinsic
dimensionality then the one previously established by PCA.
4
Scope of Thesis
In this thesis, Locally Linear Embedding and Isomap algorithms are presented as
tools for performing the analysis on the high dimensional data such as spectra,
chromaticity histograms and images. By analysis, we mean that we want to determine
how to compress this high dimensional data into a low dimensional space in which the
intrinsic qualities of the original data will correspond to the axis or dimensions of this
new, embedded data space.
Therefore, the role of LLE and Isomap techniques is examined in:
• discovering the intrinsic dimensionality of the high dimensional data

• embedding the data in question into an underlying, low dimensional space
• establishing the relationship between the original, input data, and the embedded
data
The thesis also contrasts these two methods with the well known and often used
techniques for data dimensionality estimation and embedding, chiefly, Principal
Component Analysis (PCA). In particular, the focus of this thesis is the ability of LLE
and Isomap to determine the intrinsic dimensionality of the original data and to embed
the data into a space in which dimensions correspond to the intrinsic properties of the
original data. This is contrasted by PCA's apparent inability to determine sufficiently low
intrinsic dimensionality of the input data and to associate the dimensions in the newly
formed embedded space with particular intrinsic characteristics of data present in the
original input.
The single most important result of this thesis comes in the area of spectral
reflectance analysis. Isomap and LLE were able to determine that the high dimensional
spectra data, with the original dimensionality of 101, can be reduced and embedded into a
space with the dimensionality possibly as low as 3. The advantage of being able to reduce
5
spectra to 3 dimensional space is in the fact that 3 dimensional data is much easier for
humans to visualize and manipulate.
6
Thesis Outline
Chapter 2
Provides the reader with a description of symbols and concepts used throughout
this thesis. This chapter introduces the reader to Principal Component Analysis, Root
Mean Square Error, Residual Variance and Distortion. The concepts of Chromaticity and
Color Constancy are also explained.
Chapter 3
Summarizes some of the research related to the estimation of data dimensionality,
spectra reconstruction, image classification and object recognition. Summarizes the
existing techniques used in determining the dimensionality of data and data compression
and embedding.
Chapter 4
Introduces the reader to the Locally Linear Embedding and Isomap techniques
that are central to this thesis. Provides the detailed description of algorithms. Describes
how LLE and Isomap techniques work. Demonstrates the application of LLE and Isomap
to the reader using simple geometrical examples.
Chapter 5
Presents the application of Locally Linear embedding and Isomap techniques to
the problems of determining the true dimensionality of multi-dimensional data and
embedding the high dimensional data in the low dimensional space for chromaticity
histograms and reflectance spectra. Provides the comparison with existing techniques,
such as Principal Component Analysis. Explains how the above techniques could be
7
applied to the problems of color constancy, spectra reconstruction, RGB-to-spectra
mapping and image classification.
Chapter 6
Contains a discussion of the results obtained from previous chapters. Conclusions
about effectiveness and advantages of applying Locally Linear Embedding and Isomap
techniques to the problems of color constancy, chromaticity histograms and spectra
reconstruction are drawn. Possible areas for future research are outlined.
Chapter 7
Shows the detailed algorithms that were used as a basis of computations in this
thesis. The LLE, Isomap and LLE based reconstruction algorithms are presented.
8
Chapter 2
Preliminaries
In this section of the thesis we will introduce the reader to some basic symbols,
concepts and techniques that are used throughout the thesis. This will allow the reader to
better understand both the problem domains and the scientific approach this thesis uses to
address them.
We will start by introducing the reader to some of the symbols and notations used
throughout this thesis. Next, we are going to explain some of the basic concepts in
computer vision, such as reflectance spectra, chromaticity space and color constancy. In
the third part of the introduction, we are going to explain the Principal Component
Analysis (PCA) method, which is a classical method for data dimensionality estimation
and reduction. Through the Principal Component Analysis we are also going to define
Variance, Residual Variance, the “Elbow” Technique, and show how we can use these
methods to estimate the dimensionality reduction obtained by PCA. Finally, we are going
to define the distortion, a method that can be used to compare the Isomap and PCA
methods in terms of dimensionality estimation.
9
Symbols
The following table of symbol definitions provides a summary of the notation

commonly used throughout the thesis
Table 1 - Symbols
SYMBOL DEFINITION
λ Wavelength (nm).
E (λ ) Spectral power distribution of ambient light
S (λ) Spectral reflectance of a surface
R(λ) Spectral Sensitivity of a sensor
RMS Root Mean Square (RMS) is a measure of the fit of the

approximated curve to an actual curve [18], and can be used as an
alternative measure for both color signals and tristimulus values.
Given a theoretical vector Vo and its approximation V, we can
define RMS as:
(Vo − V ) * (Vo − V )
RMS = * 100%
Vo * Vo
10
Spectra
The most accurate method we have of measuring the color behavior of objects
and light sources is to record the amount of energy present at each wavelength of visible
light. These measurements are taken using a spectrophotometer, an instrument that
divides the visible spectrum into discrete bands and records the amount of energy present
in each one. The spectrophotometers used in this thesis divide the visible spectrum into
101 bands, each 4nm wide. When we measure a color sample with a spectrophotometer,
we obtain its spectral response curve, which shows how much radiant energy the sample
transmits or reflects at each wavelength, represented as a percentage.
11
Chromaticity
An image is formed by the interaction of sensors (R), incident light (E) and
reflectance of the surface (S).
I= ∫ E (λ)R( λ) S ( λ)∂λ
Commonly, in the image analysis we would like to take the intensity of the
illumination out of this equation, so that we can make color appear more constant under
varying lighting conditions. The simplest way to do this is to move the image into
chromaticity space.
Given the R, G and B values of the pixels in an image, The International
Commission on Illumination (CIE) [15] describes the chromaticity coordinates r, g, and
b as follows:
R G B
r= g= b=
R +G + B R +G + B R +G + B
Figure 3 - Color image and its chromaticity plots
12
Color Constancy
The idea behind the color constancy is that we want to develop a vision system
that would be capable of reducing, or better yet, completely removing the effects that the
illumination has on an image. The idea for color constancy comes from the fact that our
own, human, vision system shows some form of color constancy. One of the more
obvious color constancy problems that appears in everyday life is photographing a scene
with a film that is designed for indoor images. Usually, such an endeavor produces poor
looking images. Such images introduce, for us, unexpected changes in the color to
familiar objects, such as human skin, or human faces.
In order to see why the color constancy problem occurs, we have to remind
ourselves what factors influence the overall appearance of the images taken by the
camera. The image recorded by a camera depends on three factors:
§ the physical content of the scene,

§ the illumination incident on the scene, and
§ the characteristics of the camera.
This leads to a problem for many applications where the main interest is in the
physical content of the scene, such as for example object recognition. Consider, for
example, a computer vision application which identifies objects by color. If the colors of
the objects in a database are specified for tungsten illumination (reddish), then object
recognition can fail when the system is used under the very blue illumination of blue sky.
This is because the change in the illumination affects object colors far beyond the
tolerance required for reasonable object recognition. Thus the illumination must be
controlled, determined, or otherwise taken into account.
13
Principal Component Analysis
Principal component analysis or PCA is a technique, widely used in signal

processing, statistics and computing. The basic idea in PCA is to find the components
s1,s2,...,sn so that they explain the maximum amount of variance possible by n-linearly
transformed components [16].
The basic goal in PCA is to reduce the dimension of the data. It can be proven that
the representation given by PCA is an optimal linear dimension reduction technique in
the mean-square sense. Such a reduction in dimension has important benefits. First, the
computational overhead of the subsequent processing stages is reduced. Second, noise
may be reduced, as the data not contained in the n first components may be mostly due to
noise. Third, a projection into a subspace of a very low dimension, for example two, is
useful for visualizing the data. Note that often it is not necessary to use the n principal
components themselves, since any other orthonormal basis of the subspace spanned by
the principal components (called the PCA subspace) has the same data compression or
de-noising capabilities [16]
Figure 4 - PCA Analysis. The red line shows the basis vector for the given data set.
14
The common implementation of Principal Component Analysis is through
Singular Value Decomposition (SVD).
SVD algorithm takes the covariance matrix X and produces three matrices U, D
and V such that:
X = U * D * Vt
Where
Matrix D – is a diagonal matrix of the same dimension as X with nonnegative
values along the diagonal in sorted order
Matrices U and V – unitary matrices.
We solve for the eigenvalues and eigenvectors of a square symmetric matrix (X)
with sums of squares and cross products. The result of this decomposition is shown
below. Consequently, the first and largest value in matrix D corresponds to the largest
eigenvalue and is associated with the first column in V. The second largest value in D
corresponds to the second largest eigenvalue and is associated with the second
eigenvector represented by the second column of V. Essentially, values in the matrix D
describe how much each principal component (each column in U) contributes to the
variance in the original data.
2 1 0
For example, let X = 3 2 0
 
1 3 0
By applying SVD to X we get the following decomposition
.42 − .41 − .81 5 0 0  .71 .71 0

X = .71 − .41 .58  * 0 1.73 0 * − .71 .71 0
     
.57 .82 − .12 0 0 0  0 0 1
15
As we said earlier, the goal of PCA is to reduce the dimensionality of the data.
Now we would like to know how much reduction in dimensionality occurred in the
example above. This brings us to the next topic – Residual Variance.
Variance, Residual Variance and The “Elbow” Technique
Variance is a measure of how spread out a distribution is. The variance is

computed as the average squared deviation of each number from its mean. The
mathematical formula for variance is:
σ2 =
∑ ( X − µ) 2
K
Where X is the data, K is the number of data points and µ is the mean.
Residual Variance is the amount of variance in data remaining after the first n
principal components have been accounted for. The residual variance is best explained
through the example so we will simply continue with the example and data from the
previous section.
We had
5 0 0
D = 0 1.73 0

0 0 0
The values on the diagonal are 5, 1.73 and 0. These values correspond to the
amount of variance in the first 3 components of the data X. To get the amount of variance
explained by the first n components we use the following formula:
16
∑
2
( Di )
Vn = i =1..n
∑
2
i =1..k
( Di )
Where k is the number of components and Di is the ith entry in the diagonal matrix D
The residual variance is simply
VRn = 1 − Vn
Therefore we obtain the following results:
Component 1 2 3
Di 5 1.73 0
Vi 0.8929 1 1
VRi 0.1071 0 0
Table 2 - Variance and Residual Variance for Example 1
From this data we see that first 2 components account for the 100% of the
variance. Therefore the original data X that was 3 dimensional can be reduced using PCA
to 2 dimensions. This was expected because original data’s third dimension was constant
for all three data points.
In addition to the mathematical approach, we can use a visual approach to this

problem as well. The intrinsic dimensionality of the data can be estimated by looking for
the "elbow" at which the curve showing the relationship between residual variance and
number of dimensions of data ceases to decrease significantly. More precisely, the elbow
technique is looking at the place in the graph at which adding a new dimension does not
reduce the residual error by a significant amount.
17
The arrow in Figure 6 shows the “elbow” for this particular curve.
Figure 5 – The Elbow Technique [1]. The Red arrow identifies the “elbow” in the graph.
18
Distortion
Distortion is a measure of how well a particular low-dimensional embedding

represents the original high-dimensional data.
We define distortion by the following formula [1]:
D = 1 − CorrelationCoeffiecient( H , L)
where
H = array of pair-wise distances between all the points in the original high-
dimensional space.
L = array of pair-wise distances between all the points in the embedded, low-
dimensional space.
The higher the correlation between the two data spaces, the lower the distortion will be.
Thus the best possible embedding is the one in which distortion is zero. By measuring
distortion for embeddings of different dimensionality we can plot the graph depicting
relationship between the distortion and the dimensionality of the embedding. We can then
apply the “Elbow” technique described in previous section to such graphs in order to
determine the intrinsic dimensionality of the original high-dimensional data.
19
Chapter 3
Previous Work
In this section, we are going to introduce the reader to several research areas that
are related to the problems addressed in this thesis. Firstly, we are going to talk about the
problem domains to which LLE and Isomap were applied so far. Next, we will examine
the previous work done in the area of dimensionality estimation for reflectance spectra.
Finally, we will introduce the reader to the Color and Chromaticy histograms, as these
represent an important area of research in this thesis.
Applications of LLE and Isomap
The authors of Locally Linear Embedding and Isomap view their methods as a
way of discovering compact representations for the high dimensional data. In addition to
allowing for data compaction, these methods seem to be capable of uncovering the global
structure of non-linear data. This ability to discover the non-linear properties and the
global structure of the non-linear data[1] seems to be what distinguishes the LLE and
Isomap from the traditional methods for data compaction, such as Principal Component
Analysis (PCA). Therefore, most of the original work presented by the authors of Isomap
and LLE is focused on exploiting these tools for discovering non-linear features of high
dimensional data. Several possible areas of application of LLE and Isomap to the feature
detection are examined by the authors in their respective papers. We are going to briefly
mention two of those areas.
First of those applications is the embedding and analysis of the human face
expressions. The authors claim that by embedding a number of images of human face
expressions into the three-dimensional embedding, it is possible to discover the structure
20
in the variability of the pose and expression of a human face. The experiments presented
by the authors indeed present a convincing case for the ability of LLE and Isomap to
detect these clearly non-linear properties of the human face images (see Figure 1, Chapter
1).
The other suggested application of the LLE and Isomap is semantic categorization
of words. The original LLE paper[1] demonstrates the ability of this method to embed
English words in the semantic space. In the experiment, each word is represented by a
vector counting the number of times that the word appeared in different encyclopedia
articles. After applying LLE to these word-vectors the resulting embedding showed that
the semantically similar words were grouped together.
Spectra Dimensionality Estimation
The importance of color in everyday human life cannot be overstated. The

measurements of color are present in a variety of physical and chemical measurements.
Modern image analysis cannot be even imagined without dealing in at least some aspect
with color and color measurements.
In order to represent the color adequately and precisely we need to record the
entire spectrum. However, working with the entire spectrum can be cumbersome. Firstly,
storing the color information in terms of spectra can be problematic in terms of memory.
Secondly, the problems arise in terms of complexity of computation when we are dealing
with 31 or 101 dimensional vectors of spectra. Finally, humans have problems visualizing
the data that is more then 3 dimensional. The original research in this field by Cohen[14]
and Parkkinen [17] seemed to have been motivated by computational and memory
requirements in those early stages of computing. Even though the motivation for
improving the performance and capacity requirements of color processing has somewhat
diminished over time, we are still left with the problem of representing the color
information in human comprehensible form and with dealing with complex
computational in 31 or 101 dimensional space.
21
In his paper, Cohen [14] claims that for 433 color chips taken out of the Munsell
book of Color, 99.18% of the variance depends on only three components. Furthermore
he concludes that three-component dependency may be characteristic of all the organic
pigments, including those in the retina, and thus explaining the trichromic nature of color.
Parkkinen’s research is somewhat less optimistic in terms of finding the low
number of basic components of spectra. His research based on 1257 color chips taken
from the Munsell book of color, and analyzed using the Karhunen-Loève expansion (i.e.,
Principal Component Analysis), suggests that there could be potentially eight
characteristic spectra that are needed for a good representation of all the color spectra.
Color Histograms and Image Recognition
A Color Histogram, obtained by discretizing the image colors and counting how
many times each discrete color occurs in the image, can be used as a technique for image
recognition and classification through the method called Color Indexing[8]. This view of
the role of color in image recognition stands in contrast with the classical methods, which
emphasize that geometrical cues should be viewed as a primary and most reliable factor
in image recognition. Nevertheless, the properties that the color histograms and color
indexing bring to the field of image recognition can not be ignored. Color indexing is
invariant to rotation and translation, and is quite resistant to change of view, scaling and
partial occlusion. Combined with techniques that reduce the dependence of color on the
illumination, such as different color constancy algorithms, Color Histograms offer a
powerful, processing efficient algorithm for object recognition and image classification.
22
Chapter 4
The Techniques Explained
The Isomap technique
The Isomap technique is based on the idea of viewing the problem of high
dimensionality to low dimensionality transformation as a graph problem. Isomap can be
seen as a way of generalizing Principal Component Analysis (PCA), the classic technique
for unsupervised structure discovery in high dimensional spaces.
The example of transformation performed by Isomap method is given in Figure 6.
Figure 6 - Isomap Technique [1]. Swiss roll example that demonstrates the transformation and
embedding of the 3 dimensional “Swiss Roll” into 2 dimensional space. In image A, for any two
points (dark circles) their Euclidean distance (dashed blue line) may not represent their intrinsic
similarity, as geodesic distance does (solid blue line). Image B shows two points in the Neighborhood
graph G (constructed in Step 1 of Isomap algorithm) and geodesic path (red line) between them
(constructed in step 2 of Isomap algorithm). Image C shows embedding (constructed in step 3 of
Isomap algorithm) which best preserves the shortest path distances in the neighborhood graph. The
blue line shows the straight-line distance between the two points and red line shows the geodesic
path. [included in the thesis with the written permission of Joshua B. Tenenbaum]
23
The Isomap algorithm itself consists of three steps:
1) The estimation of neighborhood graph
In this part of the algorithm, given the input points, the neighbors for each point
are found. The neighborhood for each point can be formed by either determining a set of
points that lay within some radius e of a given point, or by finding the K-nearest
neighbors[20] of a given point. Throughout this thesis the later method is used. Once the
neighbors of a given point are found, they are connected to it, forming a connected graph.
Each edge in such a graph is assigned a weight d x(i,j), corresponding to a distance
between two points.
2) Computing the shortest path graph given neighborhood graph
The geodesic distances dM(i,j) between all the connected points in the graph are
calculated by computing the shortest path between any two given data points using the
Floyd-Warshall algorithm[21]. The result of step two is a graph containing the shortest
path distances between all the K size neighborhoods. This is by far the most process
intensive part of the algorithm. However, since the computation is based on graph theory,
it is possibly to optimize this process using the Dijkstra’s algorithm for calculation of the
shortest path [1][19].
3) Construction of lower dimensional embedding
In the final step of the algorithm, d-dimensional Euclidean space embedding of

the data is constructed preserving the original data intrinsic geometry. The embedding is
based on the method of Multidimensional Scaling (MDS) [22]. However, unlike the
classical MDS, the MDS used for Isomap algorithm is applied to geodesic distances.
24
The LLE Technique
Similar to the Isomap, the LLE technique consists of the three basic parts that
correspond to the Isomap algorithm steps.
Figure 7 - LLE Technique. Three steps of the LLE technique. In step 1 of the algorithm K nearest
neighbors are found for each point. In step 2 weights Wij are caclulated. These weights describe the
position of point Xi in relationship to its neighboring point Xj. In step 3 embedding in low
dimensional space is performed using the weights Wij obtained in the step2.
25
The LLE steps are the following:
1) Neighborhood grouping
Find the neighborhood for any given point in the starting space. This can be done
using the technique such as K nearest neighbors[20].
2) Calculation of weights
In a second step, we want to express the relationship between any given point in
the original space and its neighbors. For that we use reconstruction weights Wij. We
simply construct a matrix whose columns correspond to every point of the original data
space and whose number of rows corresponds to the number of neighbors. Then, for
every row we calculate the weights that describe the position of the given data point
compared to its neighbors. The weights Wij that best reconstruct the point Xi from its
neigbours X j are computed by minimizing the solution to the following equation:
ξ (W ) = ∑ X − ∑Wi ij Xj
i j
3) Embedding
Finally we find the embedding of the original data set in the low dimensional
embedding space. This is again done by trying to preserve the geometric properties of the
original space through the minimization of the error given by the equation:
ξ (Y ) = ∑ Yi − ∑ WijY j
i j
Where W’s are weights computed in step 2, Yi and Yj are the coordinates of the point and
its neighbors in the embedded space.
26
LLE vs. Isomap
The biggest difference between the Isomap and Locally linear Embedding
algorithm is in the fundamental approach to the problem of dimensionality reduction and
data representation. Isomap sees the dimensionality reduction problem as a graph
problem, in which data are represented as connected graphs and relationship among data
is described through the use of geodesic distances. On the other hand, LLE takes a
different approach, looking at this problem from purely geometrical perspective.
However, both the LLE and Isomap are concerned with preserving the neighborhoods
and their geometric and graph relationships, respectively.
One important advantage of the Isomap algorithm is that, as a part of its
processing, it calculates the relationship between the dimensionality and distortion. This
can be very useful in estimating the intrinsic dimensionality of high dimensional data
being processed. The intrinsic dimensionality is also important in determining the size of
target embedded space for the data being processed. An additional advantage of Isomap
algorithm is that it does not require that number of neighbors be greater then the number
of dimensions in original data space.
This is an important requirement of the LLE algorithm, because in order to solve
for reconstruction weights in step 2 of the algorithm (see page 34), we must have at least
the same number of neighbors as there are dimensions. If this condition does not hold the
system of equations we are trying to solve will have more variables then there are
equations. This is a quite severe limitation given that data such as images potentially
could have a huge number of dimensions. Therefore, if we were to process images, we
would require massive input data sets.
In addition, what makes Isomap algorithm more practical in terms of processing
time is that it can be optimized using Dijkstra’s algorithm for computation of shortest
paths in a graph[19]. This is very significant, because this optimization reduces the
processing time from the order of hours to the order of minutes. Such a reduction in time
is explainable by observing that the time complexity for the Dijkstra’s algorithm is
O(VlgV + E)[19] while the time complexity for the Floyd-Warshall algorithm is O(V3)
[21].
27
Chapter 5
The application of LLE and Isomap
Chromaticity Histograms
Color histograms and chromaticity histograms have been used widely in object
recognition [8] [10] and illumination estimation [5] [11]. This section presents the use of
Isomap method in analyzing the structure of the space of binarized chromaticity
histograms.
Input Data
A chromaticity histogram of an image is a count of the number of pixels occurring

in the image as a function of the pixels’ chromaticities. A binarized chromaticity
histogram is simply a chromaticity histogram in which the histogram bin counts have
been set to zero or one depending on whether the original count was above or below a
small threshold. A ‘1’ in a binarized chromaticity histogram indicates the presence of
that chromaticity in the image. Thresholding is commonly used to avoid the problem of
noise causing spurious chromaticities to appear in the histograms. For this analysis, we
tried both rg and rb chromaticity spaces, where
r=R/(R+G+B);
g=R/(R+G+B);
b=R/(R+G+B)
28
R, G, and B are raw digital camera output values.
We used a library of 1050 images taken using different digital cameras, under
different lighting conditions, both indoors and out, and including, for example, portraits
of people, scenes from nature, Macbeth charts, buildings, cars.
Figure 8 - Sample of Image Library used for Chromaticity Histogram experiment.
29
Each image contains a standard grey card somewhere within it for use in
evaluating its color cast. The images were also taken using a variety of settings on the
cameras in terms of flash or no flash, automatic white balance on or off, manual white
balance set to tungsten or daylight, and so forth. The cameras did not have a gamma
option, so we presume some gamma-like tone correction was applied to all images.
Before generating an image’s chromaticity histogram, the image is first pre-

processed to remove very bright or clipped pixels (i.e., R or G or B near to the camera
maximum) and very dark pixels. Both clipped and dark pixels lead to unreliable
chromaticity data. The chromaticity space (a triangle with vertices at the origin, (0,1) and
(1,0)) is coarsely quantized into 450 distinct chromaticity values. The resulting
histograms therefore also contain 450 bins. We can think of the histograms as points in a
450-dimensional space. Although in principle all 450 bins could get used, in practice only
251 of them actually do get used by at least one of the 1050 images. Hence, the effective
dimensionality of the space of chromaticity histograms of the entire set of images is only
251. Table 3 shows an example chromaticity histogram.
Table 3 - Binary Chromaticity Histogram
G - component bins
Bin # 1 2 3 4 . . . 22 23
1 0 0 0 0 0 1 1 1 1
2 1 0 0 0 0 1 0 0 0
R - component bins
3 1 0 1 1 0 0 0 0 1
1 0 1 0 0 0 0 0 1
0 0 1 0 1 0 1 0 0
. 1 0 0 0 0 0 0 1 1
. 0 1 0 0 0 1 1 0 1
22 0 0 1 0 0 0 0 1 0
23 0 0 1 0 1 1 0 1 0
30
Determining Dimensionality
The hypothesis was that the chromaticity histogram data might have a much lower
dimensionality then the initial 251 dimensions of the input space. In addition, we
speculated that the main dimensions might correspond to variations in image colorcast
caused by scene illumination conditions and camera white balance settings.
We processed the chromaticity histogram data using the Isomap algorithm and
calculated distortion for the first 10 dimensions of the embedding. As a result we were
able to establish the relationship between the distortion and the sub-space dimension, as
shown in Figure 10. Using the the “Elbow” technique on Isomap’s distortion data we
concluded that the histograms fall in a subspace of 5 to 6 dimensions.
In order to put the obtained dimensionality estimation in perspective we also
calculated the dimensionality estimation using the Principal Component Analysis based
on both the Residual Variance and Distortion. As the Figures 9 and 10 show, the
dimensionality estimate based on Principal Component Analysis is significantly higher
then the estimate obtained using the Isomap technique. In fact, the PCA’s dimensionality
estimate based on the Residual Variance (Figure 9) does not identify the “elbow” in the
first 10 dimensions at all.
31
Table 4 - Residual Variance for 1050 Chromaticity Histograms based on PCA technique
Dim 1 2 3 4 5 6 7 8 9 10
Res. 0.8527 0.7348 0.6647 0.6140 0.5679 0.5364 0.5137 0.4929 0.4747 0.4579
Var.
Figure 9 - Residual Variance for 1050 Chromaticity Histograms based on PCA technique
32
Table 5 - Distortion for 1050 Chromaticity Histograms based on Isomap technique
Dim. 1 2 3 4 5 6 7 8 9 10
Dist. 0.6162 0.3280 0.2155 0.1688 0.1422 0.1204 0.1137 0.1058 0.1015 0.0976
Table 6 – Distortion for 1050 Chromaticity Histograms based on PCA technique

Dim 1 2 3 4 5 6 7 8 9 10
Dist. 0.7265 0.5751 0.4884 0.3981 0.3199 0.2894 0.2656 0.2498 0.2431 0.2337
Figure 10 - Distortion for 1050 Chromaticity Histograms based on PCA and Isomap techniques.
Isomap graph is labelled with stars, while the the PCA graph is labelled with circles.
33
Embedding
To visualize the nature of the embedding we have plotted thumbnails of the

images at the embedding location of their histograms. Figures 11-13 show the first 3
dimensions of the embedding of the histograms. In either representation (especially with
larger and more numerous slices), there appears to be a clear variation in color cast along
dimensions 2 and 3. Roughly speaking, dimension 2 reflects an orange-to-cyan variation.
This corresponds to a variation in red versus blue for a roughly constant amount of green.
Similarly, dimension 3 reflects a change from cyan to purple. This corresponds to a
variation in red versus yellow. We were somewhat surprised that colorcasts do not
appear to be the dominant aspect of dimension 1. Our impression is that dimension 1
reflects the number of non-zero histogram bins. In other words, the primary variation
along dimension 1 appears to be the number of distinct colors in the image. Figures 14-16
show the dimensions 2, 3 and 4 of the embedding of the histograms. Figures 17-19 show
the dimensions 3, 4 and 5 of the embedding of the histograms. Figures 20-22 show the
dimensions 4, 5 and 6 of the embedding of the histograms. Figures 23-25 show the
dimensions 1, 3 and 5 of the embedding of the histograms.
34
Figure 11 - Each of the 4 panes shows a slice orthogonal to dimension 3 of the subspace embedding
with subspace dimension 1 varying vertically and 2 horizontally.
35
Figure 12 - Similar to Figure 11 except each of the 4 panes shows a slice orthogonal to dimension 2 of
the subspace embedding with subspace dimension 1 varying vertically and 3 horizontally.
36
Figure 13 - Similar to Figure 11 except each of the 4 panes shows a slice orthogonal to dimension 1 of
the subspace embedding with subspace dimension 2 varying vertically and 3 horizontally.
37
with subspace dimension 2 varying vertically and 3 horizontally
38
39
40
41
42
43
44
45
46
47
48
49
Image Classification
Locally Linear Embedding and Isomap techniques can be applied to image

classification and grouping problems. As with any other image recognition or
classification technique, LLE and Isomap face the problem of color constancy. As we
stated in Chapter 2, the color constancy problem can be summarized as the problem of
varying image appearance caused by the varying illuminant. Our goal is to use the
methods that are capable of reducing, or better yet, completely removing the effects that
the illumination has on an image.
The approach used in this thesis to address the color constancy problem is the
one of using chromaticity histograms, described in previous sections (see page 37). By
using the chromaticity histograms, we can eliminate the impact of the lightning
conditions, and in particular the light intensity, on the images, as explained in Chapter 1
Figure 26 - Image Grouping. Indoor vs. Outdoor
50
Figure 27 - Image Grouping. Indoor vs. Outdoor
In Figures 26 and 27, we can see how similar types of images get grouped
together in the embeddings. In particular in the Figure 27, the images taken indoors are
grouped somewhat loosely together in the left upper corner of the plot. At the same time
the images taken outdoors are grouped in the lower right corner. Similarly, in the Figure
27, it is possible to distinguish clearly between the two groups of images. Images of the
fruits and vegetables are located in the upper half of the plot, while the images of scenes
of nature are grouped at the bottom.
Image groupings and classifications in the above two examples are based on the
chromaticity histograms. This however means that the position of the image in the
embedding is determined primarily by the color content of the image. Thus, even if we
have 2 images of two totally different objects that share a significant number of colors,
they would still be placed in the similar location in the embedding.
To address this problem, the location of each RGB value in the image has to be
taken into account. In essence, every single pixel in the image must be treated as a
separate dimension. This means that even for a relatively small image the number of
dimensions would easily reach tens of thousands.
51
For example, if a 300 by 200 image needs to be processed, the original
dimensionality would end up being 60,000 assuming that we encode the color as a single
number.
This obviously would mean that the LLE technique, which needs neighborhoods
of size greater then the number of dimensions, would not work because we would need a
huge number of sample images to start with, and the processing would take too long for
any practical purpose. The Isomap technique offers hope in this case, as the Isomap’s size
of neighborhood does not depend on the original data dimensionality. In addition, Isomap
offers itself to optimization in terms of processing time, though the use of optimized
graph based algorithms, such as Dijkstra’s algorithm that can reduce the processing time
to O(VlgV + E)[19], where V is the number of vertices corresponding to the number of
data points in the original space and E is the number of edges corresponding to the
number of neighborhood relationships between the data points.
52
Conclusion
Chromaticity histograms lie in a subspace of quite low dimension, with some of

the primary dimensions reflecting the overall color cast of the corresponding image. The
number of dimensions in the subspace could be possibly as low as 5 or 6. LLE and
Isomap methods can be used on chromaticity histograms to provide the image
classification and grouping analysis.
53
Spectra
Spectra are typically hard to work with and difficult to visualize due to their high
dimensionality. Typically, spectra are represented as 31 or 101 dimensional vectors,
depending on the range and sampling rate. The previous research, based on the use of
PCA and its derivations [4] [6] [7] [23] established that the underlying spectra
dimensionality is much lower then the 31 or 101 dimensions. Depending on the method
and data sets used, the spectra’s underlying dimensionality was estimated to be between 3
and 8. In this section, we explore how LLE and Isomap are going to fare in the domain of
spectra reflectances and illuminant spectra compared to PCA based analysis.
Input Data
Three sets of input data[3][9] were considered:
a) Kodak spectra - consisting of 170 spectra taken of the various objects including:
rocks, trees, plants and skin.
b) Combined spectra - including 1996 spectra samples that among other spectra libraries
includes, Krinov[12], Kodak[13] and Munsell spectra.
c) Illuminant spectra – consisting of 102 illuminant spectra.
For all three these spectra databases, each individual spectrum consists of 101
data points corresponding to measurements made in the 380-780 nm range sampled at
every 4nm. This sampling rate effectively defined the dimensionality of the input data at
101 dimensions.
Unlike with the histograms, no prior pre-processing is necessary with the spectra
samples.
54
Dimensionality Estimation
The hypothesis was that the spectra data might have a much lower dimensionality
then the initial 101 dimensions of the input space.
We processed the spectra data using the Isomap algorithm and caclulated
distortion for the first 10 dimensions of the embedding. As a result we were able to
establish the relationship between the distortion and the sub-space dimension, as shown
in Figure 31, 32 and 33. Using the the “Elbow” technique on Isomap’s distortion data we
concluded that the spectra fall in a subspace of 3 to 4 dimensions.
In order to put the obtained dimensionality estimation in perspective we also
calculated the dimensionality estimation using the Principal Component Analysis based
on both the Residual Variance and Distortion. As the Figures 28-30 and 31-33 show, the
dimensionality estimate based on Principal Component Analysis is very close or slightly
lower then the one of Isomap technique.
55
Residual Variance (PCA)
Residual Variance in PCA is a classical methods of estimating the dimensionality of data.
For 1996 reflectance spectra the following residual variance is obtained using the PCA
technique:
Table 7 - Residual Variance for 1996 Reflectance Spectra based on PCA technique
Dim 1 2 3 4 5 6 7 8 9 10
Res. 0.2838 0.1420 0.063 0.0277 0.0126 0.0078 0.0058 0.0041 0.0029 0.0021
Var.
Figure 28 - Residual Variance for 1996 Reflectance Spectra based on PCA technique
By applying the Elbow technique to the Figure 28 and the data in Table 7 we estimate
that the dimensionality of the combined 1996 spectra samples based on Residual
Variance falls between 4 and 5.
56
For 170 Kodak spectra the following residual variance is obtained based on PCA
technique:
Table 8 - Residual Variance for 170 Kodak Reflectance Spectra based on PCA technique
Dim 1 2 3 4 5 6 7 8 9 10
Res. 0.1977 0.0771 0.0365 0.0176 0.0104 0.0059 0.0033 0.002 0.0012 0.0008
Var.
Figure 29 - Residual Variance for 170 Kodak Reflectance Spectra based on PCA technique
that the dimensionality of the combined 170 Kodak spectra samples based on Residual
Variance falls between 4 and 5.
57
For 102 Illuminant spectra the following residual variance is obtained:
Table 9 - Residual Variance for 102 Illuminant Spectra based on Isomap technique
Dim. 1 2 3 4 5 6 7 8 9 10
Res. 0.0186 0.0058 0.0016 0.0004 0.0004 0.0004 0.0004 0.0004 0.0004 0.0004
Var.
Figure 30 - Residual Variance for 102 Illuminant Spectra based on PCA technique
that the dimensionality of the combined 102 Illuminant spectra samples based on
Residual Variance is 3.
58
Distortion (PCA and Isomap)
Distortion measurement allows us to directly compare a dimensionality estimate obtained

by PCA with the one obtained through Isomap.
For 1996 reflectance spectra the following distortion is obtained:
Table 10 - Distortion for 1996 Reflectance Spectra based on PCA technique

Dim 1 2 3 4 5 6 7 8 9 10
Dist. 0.1588 0.0488 0.0101 0.0035 0.0005 0.0002 0.0001 0.0001 0.0 0.0
Table 11 – Distortion for 1996 Reflectance Spectra based on Isomap technique

Dim. 1 2 3 4 5 6 7 8 9 10
Dist. 0.2855 0.0483 0.0201 0.0111 0.0110 0.0103 0.0116 0.0117 0.0121 0.0125
Figure 31 - Distortion for 1996 Spectra based on PCA and Isomap techniques. Isomap graph is
labelled with stars, while the the PCA graph is labelled with circles.
59
For 170 Kodak spectra the following distortion data is obtained:
Table 12 - Distortion for 170 Kodak Reflectance Spectra based on PCA technique
Dim. 1 2 3 4 5 6 7 8 9 10
Dist. 0.1588 0.0488 0.0101 0.0035 0.0005 0.0002 0.0001 0.0001 0.0 0.0
Table 13 - Distortion for 170 Reflectance Spectra based on Isomap technique

Dim. 1 2 3 4 5 6 7 8 9 10
Dist. 0.0832 0.0204 0.0097 0.0067 0.0055 0.0046 0.0050 0.0042 0.0042 0.0043
Figure 32 - Distortion for 170 Kodak Reflectance Spectra based on PCA and Isomap techniques.
Isomap graph is labelled with stars, while the the PCA graph is labelled with circles.
60
For the 102 Illuminant spectra, Isomap method produces the following distortion:
Table 14 - Distortion for 102 Illuminant Spectra based on PCA technique

Dim. 1 2 3 4 5 6 7 8 9 10
Dist. 0.0014 0.003 0.0001 0.00001 0.0 0.0 0.0 0.0 0.0 0.0
Table 15 - Distortion for 102 Illuminant Spectra based on Isomap technique

Dim. 1 2 3 4 5 6 7 8 9 10
Dist. 0.0015 0.0003 0.0001 0.0001 0.0001 0.0001 0.0001 0.0001 0.0001 0.0001
Figure 33 - Distortion for 102 Illumination Spectra based on PCA and Isomap techniques. Isomap
graph is labelled with stars, while the the PCA graph is labelled with circles.
61
From the PCA and Isomap distortion graphs above we can see that for all three
spectra libraries Isomap’s “elbow” occurs at about the same dimension as PCA’s
“elbow”. More precisely, both the Isomap’s and PCA’s distortion graphs indicates that
the “elbow” occurs somewhere in between dimensions 3 and 4. Thus, based on the
distortion measurements Isomap and PCA indicate the dimensionality for the spectra data
to be between 3 and 4.
62
Embedding
In order to visualize the data we are processing using LLE and Isomap we need to
embed the original data in the low dimensional space. As we have concluded that spectra
data can be reduced to 3 to 4 dimensions, the best was to embed the data from the
perspective of visualizing the data would be to use a 3 dimensional embedding.
The problem with 3D embedding is that it is hard to visualize such an embedding
without some tools for allowing rotation and zooming, so for the purposes of this thesis
we will be presenting the 2D slice of the 3D spectra data as viewed from the direction of
the first axis.
In addition to just simply embedding the data points obtained using the Isomap
and LLE into 2 or 3 dimensional space, we would like to assign the RGB values to each
point based on its corresponding spectrum. This is possible since both LLE and Isomap
preserve the ordering between the original data and the embedded data, so for each
embedded point we can find its corresponding input spectrum value. Given spectra values
we can easily calculate RGB values for a data point by integrating spectra with the
camera or retina sensor spectral sensitivities. For this particular experiment we have
chosen to do spectra-to-RGB transformation by integration using the Sony DCX-930
camera sensor sensitivities.
63
Isomap based embedding
The following figures illustrates the embedding of the 1996 and 171 spectra input
data using the Isomap technique:
Figure 34 - 1996 Spectra embedded in 2D space using Isomap
Figure 35 - 170 Spectra embedded in 2D space using Isomap
64
LLE based embedding
The following figures illustrate the embedding of the 1996 and 171 spectra input
data using the LLE technique:
Figure 36 - 1996 Spectra embedded in 2D space using LLE
Figure 37 - 170 Spectra embedded in 2D space using LLE
From the Figures 34 – 37 we can see that LLE and Isomap methods produced
somewhat different embeddings in terms of positioning of the particular spectra. This can
be explained by noting that LLE and Isomap preserve different qualities of the original
data while performing the embedding. LLE preserves the Euclidean geometric distances
between the input point and its neighbors, while Isomap preserves the geodesic distances
among the input data. Although the embeddings are different, we can see that both
methods produced embeddings with the spectra of similar RGB values grouped together.
65
Therefore, both these embeddings preserved the intrinsic qualities of the original data.
66
Spectra Reconstruction and Error Estimation
Once the embedding is found using either the Isomap or LLE, we can try to
estimate how representative this embedding is of the actual data. The Distortion method
described in the Chapter 2 gives us one way of estimating the relationship between the
embedding and the original data. The other way to see how representative the embedding
is to reconstruct the original data based on the embedding. In essence, using the
embedding and the fact that LLE and Isomap methods preserve the localized
characteristics of the data, it is possible to reconstruct the original data.
Given the embedding and original data set we can construct the embedded-based
estimation of the original data set by (See LLE reconstruction algorithm in the Appendix
A):
1) Finding the neighbors for each embedded data point.

2) Finding the weights describing the position of the embedded data point in relation to
its neighbors.
3) Estimate the original data points by reconstructing them given the weights and
neighbors found in the embedded data space in steps (1) and (2).
After the estimated original data is found, we can calculate the root mean square
error (RMSE) for each data point based on the original data set and the estimation we
made. Given the RMSE for each data point we can calculate the mean and median RMSE
for the data set. Mean and median RMSE can then be used as a basis of comparison with
other reconstruction methods, such as PCA.
As input data for the reconstruction, we will use the 170 Kodak spectra described
in the previous section. Given that 3 dimensions are determined to be adequate to
represent the spectra, the 3 dimensional embedded space will be used for basis of
67
reconstruction. We will use the LLE based reconstruction on both the embeddings
obtained using LLE and Isomap.
The errors for the LLE constructed embedding are the following:
Mean RMSE = 0.0305

Median RMSE = 0.0166
The errors for the Isomap constructed embedding are the following:
Mean RMSE = 0.0434

To put these results into perspective, the PCA based reconstruction can be applied
to the same data set. The PCA based reconstruction is based on the idea of finding 3 basis
vectors (B) that can then be used to reconstruct the original spectra data set.
The first step is to subtract the mean from data effectively centering the data
around zero. Then the basis vectors can be found using the Singular Value
Decomposition (SVD) method. SVD decomposes the input matrix X, into matrices U, D,
and V such that:
X = U * D * Vt
Where the columns of the matrix V represent the basis vectors (See Chapter 2, pages 23-
24).
Given the basis vectors, we can solve the following systems of equations for W’s for
each spectra data point:
68
R = W1* Sr *B 1 + W2* Sr * B2 + W3* Sr *B3
G = W1* Sg *B 1 + W2* Sg *B 2 + W3* Sg *B 3
B = W1* Sb *B 1 + W2* Sb *B2 + W3* Sb *B 3
where Sr, Sg, Sb are sensor sensitivity vectors; R,G and B are RGB values for a given
spectra; B1, B 2 and B 3 are basis vectors
By obtaining weights W, we can then reconstruct the spectra by simply substituting

values in the following formula:
SP = W1* B1 + W2* B2 + W3* B3
for all the weights.
Then the RMSE can be calculated based on the reconstructed spectra and the original
data set:
Mean RMSE = 0.0418

Finally, we can also reconstruct the spectra values given their respective RGB values
using the simple interpolation on the RGB values. Given the RGB values of the original
spectra set and original spectra set we can construct the RGB-based estimation of the
original data set by
1) Finding the neighbors for each RGB data point.

2) Finding the weights describing the position of the RGB data point in relation to its
neighbors.
3) Estimate the original spectra data points by reconstructing them given the weights and
neighbors found in the RGB data space in steps (1) and (2).
69
As in two previous cases we used 170 Kodak spectra. Spectra’s RGB values were
obtained by integrating the spectra with sensors of Sony DCX-930 camera.
The results for the RGB based reconstruction are the following:
Mean RMSE = 0.05

From the data obtained in all three types of spectra reconstruction we can see that
PCA based reconstruction produced RMSE that is close to the LLE based reconstruction
error, thus validating the LLE based reconstruction technique. In addition, we can see the
LLE’s reconstruction is about 25% better then PCA reconstruction and almost 40% better
then RGB based reconstruction.
70
RGB-to-Spectra Mapping
We have seen in the previous sections that if we are given a particular set of
spectra we can produce a low dimensional embedding of such a spectra using the LLE or
Isomap methods. Furthermore, we can assign the embedded spectra their corresponding
RGB values, as shown in Figure 38. In essence, this allows us to establish a mapping
between the RGB values in the low dimensional space and the spectra in high
dimensional space.
We also wanted to see if it is possible to establish the relation that would go in
other direction, from a low to high dimensional space. Such a mapping is interesting
because it allows us to predict the spectra values given the RGB values, effectively
allowing us to quickly go from low dimensional to high representation and thus gain in
the amount of information we posses. Assuming that this mapping is unique, or at least
close to unique, we can then use the mapping to look up the values of spectra given the
values in RGB space.
However, as we can see in the Figure 38, since the original spectra data was
sparse (consisting of only 170 spectra data points), the resulting embedding is sparse as
well. If we are to establish an effective RGB-to-Spectra mapping we need to either have
more spectra data to begin with or to somehow reconstruct the missing values.
Figure 38 - Embedding of 170 reflectance Spectra
71
In this thesis we are going to assume that we do not have additional spectra data
available to us. Therefore, we are going to focus on reconstructing the missing values.
The reconstruction is based on the linear interpolation in the embedded space. We

take the neighbors of a data point we are looking for and calculate the weights that
describe the data point’s position in relation to its K closest neighbors. Once we have the
weights and neighbors for a given point, we simply move to spectra space and interpolate
the point we are looking for given its neighbors and weights that we calculated in the
embedded space.
By repeating this process for all the data points that are missing we can
reconstruct all the points in the embedded space, and construct the color map, as seen in
the Figures 39-41:
Figure 39 - Reconstructed RGB values of Spectra embedded in 2D space - low resolution
72
Figure 40 - Reconstructed RGB values of Spectra embedded in 2D space - medium resolution
Figure 41 - Reconstructed RGB values of Spectra embedded in 2D space - high resolution
Given the above color map, we can run the color histogramming algorithms on it
in order to see how many unique colors are in the color map. The color histogram can be
helpful in a sense that it will show us how unique mapping is between the RGB and the
spectra space. As expected, the RGB-to-spectra mapping is not unique; however, the
number of duplicates is below 5 %. The number of duplicates is not high enough to
prevent us from establishing an approximately one-to-one relationship between the RGB
and spectra space. This means that an effective mapping can be established between the
73
two data spaces. This mapping will be a lookup table showing the mapping between the
RGB and spectra values.
We can use this mapping to predict the values in the spectra space based on the
data in RGB space and the other way around. In practical terms, if we are given a RGB-
to-spectra mapping and some new, previously unseen RGB value X, we can predict the
value of spectra for RGB value X. In, essence all we have to do is perform a lookup in
our RGB-to-spectra mapping table and find the spectra associated with out given RGB
value.
Error estimation for RGB-to-Spectra mapping
If we are to predict the values of spectra given the RGB-to-Spectra mapping, we

would like to know what kind of error we might be facing. In order to estimate the error
we are going to do the following:
1) We will take our spectra input data and split it into two sets of equal size. We
will take the first set and produce the 3 dimensional LLE embedding.
2) We will then calculate the RGB values for the embedded data points using the
original spectra.
3) Using interpolation we will construct RGB-to-Spectra mapping based on the
first data set as described in previous section (Figures 39-41).
4) For every spectra in the second set we will calculate its RGB value and use it
to predict the spectra value through the RGB-to-Spectra map obtained in step
3.
5) Finally, we will compare the predicted spectra values with actual ones and
measure the difference using RMSE
74
For the RGB-to-Spectra mapping of 170 Kodak spectra obtained by LLE the
errors are as follows:
Mean RMSE = 0.0557

We can measure the same error for the Isomap based RGB-to-Spectra mapping by
repeating the previous 5 steps, except that is step 1 we would do Isomap based
embedding. The errors are as follows:
Mean RMSE = 0.0574

Finally, to put these errors in perspective we can use PCA based spectra
reconstruction to calculate the errors. The idea behind the PCA based reconstruction is
that we can use the set of basis vectors (B) obtained from a known spectra data set to
predict the values of some previously unknown spectra.
To obtain a meaningful measurement of how well PCA can be used to predict the
spectra values given their RGB’s we will take a set of spectra and split it into two sets.
We can then use the first set to calculate the first 3 basis vectors. Then we can use these 3
basis vectors to solve the following system of equations for W’s for each spectra point
belonging to the second set:
R = W1* Sr *B 1 + W2* Sr * B2 + W3* Sr *B3

G = W1* Sg *B 1 + W2* Sg *B 2 + W3* Sg *B 3
B = W1* Sb *B 1 + W2* Sb *B2 + W3* Sb *B 3
where Sr, Sg, Sb are sensor sensitivity vectors; R,G and B are RGB values for a given
spectra; B1, B 2 and B 3 are basis vectors
75
By obtaining weights W, we can then reconstruct the spectra of the second set by
simply substituting values in the following formula:
SP = W1* B1 + W2* B2 + W3* B3
for all the weights.
Then the RMSE can be calculated based on the reconstructed spectra and the
original spectra of the second data set:
Mean RMSE = 0.0477

From the errors obtained above we can see that the RGB-to-Spectra mapping can
be used to performs the effective lookups for missing spectra and that the errors produced
by these lookup are close to the errors produces by PCA based spectra reconstruction
method.
76
Conclusion
Reflectance and Illumination spectra lie in a non-linear subspace of dimension

possibly as low as 3. LLE and Isomap algorithms can be used to produce spectra
embedding in three-dimensional space that show spectra being grouped according to their
respective RGB values. It is also possible to produce a mapping between RGB and
spectra that although not entirely unique, can be used to predict spectra given RGB
values.
77
Chapter 6
Discussion
This thesis examines the use of two relatively new techniques, Locally Linear
Embedding and Isomap, for the analysis and transformation of high dimensional data.
Throughout the thesis comparisons are made with the existing, and widely used
techniques with similar purpose, such as Principal Component Analysis. This thesis is
limited to analysis of LLE and Isomap techniques when they are applied to the domains
of Computer Vision: images, chromaticity histograms and spectra.
The thesis illustrates how LLE and Isomap can be applied to reflectance spectra
analysis. The major focus in the reflectance spectra analysis application is to determine
the true dimensionality of the reflectance spectra. Previous research by Parkkinen has
shown that the true dimensionality of spectra can be as low as 3. This thesis shows that
LLE and Isomap techniques estimate that dimensionality to be between 3 and 4.
Analysis of the Chromaticity Histograms is also performed using the LLE and
Isomap. These techniques were able to discover the underlying dimensionality of the
chromaticity histograms, while the Principal Component Analysis was not successful in
establishing any reasonably low dimensionality estimate. Thus, starting with the 251
dimensional chromaticity histogram data, it was possible to reduce it to 5 or 6
dimensions. This allows the chromaticity histogram data to be embedded in a 5 to 6
dimensional space. Such a 5 to 6 dimensional embedding can then be analyzed and
conclusions drawn about what each particular dimension in the embedded space stands
for. Furthermore analysis of the image space can take place and the conclusions about
image groupings and classification can be drawn.
78
The RGB-to-spectra mappings are also examined in this thesis. It is possible to
establish the mapping between the RGB values embedded in the 3 dimensional
embedding space and spectra in 101 dimensional space. This mapping is not entirely
unique, but the amount of the overlap is not thought to be significant. These mappings
allow us to move from one space to another with ease. For example, given a RGB-to-
spectra mapping and some new RGB value, we can predict its corresponding spectra
value.
The major area of the future work lays in the analysis and processing of images
directly, without going to image histogram space first. This would allow the images to be
analyzed not on the color content alone, but also on the physical content of the scenes
these images represent. In order for the processing of the images to take place, it is
important to discover the appropriate way of encoding the images, so that efficient
processing of such encoding is possible.
79
Chapter 7
Appendix A
LLE Algorithm
1. Find neighbors in X space

for i=1:N
compute the distance from Xi to every other point Xj
find the K smallest distances
assign the corresponding points to be neighbors of Xi
end
2. Solve for reconstruction weights W.

for i=1:N
create matrix Z consisting of all neighbors of Xi
subtract Xi from every column of Z
compute the local covariance C=Z'*Z
solve linear system C*w = 1 for w
set Wij=0 if j is not a neighbor of i
set the remaining elements in the ith row of W equal to
w/sum(w);
end
3. Compute embedding coordinates Y using weights W.

create sparse matrix M = (I-W)'*(I-W)
find bottom d+1 eigenvectors of M
(corresponding to the d+1 smallest eigenvalues)
set the qth ROW of Y to be the q+1 smallest eigenvector
(discard the bottom eigenvector [1,1,1,1...] with eigenvalue
zero)
80
IsoMap Algorithm
1 Find the K nearest neighbors for all points

for i=1:N
compute the distance Wij from Yi to every other point Yj
set YiYj edge’s weight to Wij
find the K smallest edges
set all other edges for Yi to infinity
2 Find the geodesic distance for all the pairs of point sin the graph
for i=1:N
Compute the geodesic distance between Yi and all Yj using
Floyd's algorithm
For k=1:N
If Wij > Wik + Wkj then Wij= Wik + Wkj
3 Construct the embedding of d-dimensional Euclidean space
Apply the classical multidimensional scaling algorithm to graph obtained in step 2
81
LLE Reconstruction Algorithm
1 Create embedding Y of data X using the LLE algorithm

2 Find neighbors in embedded space Y.
for i=1:N
compute the distance from Yi to every other point Yj
find the K smallest distances
assign the corresponding points to be neighbors of Yi
end
3. Solve for reconstruction weights W in embedded space Y.
for i=1:N
create matrix Z consisting of all neighbors of Yi
subtract Yi from every column of Z
compute the local covariance C=Z'*Z
solve linear system C*w = 1 for w
set Wij=0 if j is not a neighbor of i
set the remaining elements in the ith row of W equal to
w/sum(w);
end
4. Reconstruct points in space X based on neighborhoods and weights of space Y
for i=1:N
create matrix Z consisting of all neighbors of Xi
multiply Z with to obtain reconstructed X; Xirec=Z*Wi
end
82
References
1. Joshua B. Tenenbaum, Vin de Silva, and John C. Langford, A Global Geometric

Framework for Nonlinear Dimensionality Reduction, Science, Volume 290, pages
2319-2323, Dec 22 2000
2. Sam T. Roweis and Lawrence K. Saul, Nonlinear Dimensionality Reduction by

Locally Linear Embedding, Science, Volume 290, pages 2323-2326, Dec 22 2000:
3. Kobus Barnard, Lindsay Martin, Brian Funt, and Adam Coath, A Data Set for Color
Research, Color Research and Application, (to appear).
4. G. Finlayson, M. Drew and B. Funt, Color Constancy: Generalized Diagonal

Transforms Suffice , J. Opt. Soc. Am. A, 11(11), pages 3011-3020, 1994a.
5. B. Funt, V. Cardei and K. Barnard, Method of Estimating Chromaticity of

Illumination Using Neural Networks, U.S. Patent 5,907,629.
6. D.B. Judd, D.L. MacAdam and G.W. Wyszecki, Spectral distribution of typical
daylight as a function of correlated color temperature, J. Opt. Soc. of Am. A, Vol. 54,
1031, 1964.
7. L. Maloney and B. Wandell, Color constancy: a method for recovering surface

spectral reflectance, J. Opt. Soc. Am. A, 3:29-33, 1986.
8. M.J Swain and Ballard, Color Indexing, Intl. Journal of Computer Vision, 7:1, pages
11-32, 1991.
9. Computational Vision Lab, Simon Fraser University, Data for Computer Vision and
Computational Colour Science (http://www.cs.sfu.ca/~colour/data/)
83
10. Funt, B.V., and Finlayson, G., Color Constant Color Indexing, IEEE Transactions on
Pattern Analysis and Machine Intelligence, 17 (5), pages 522-529, May 1995
11. G. Finlayson, P. Hubel and S. Hordley, Color by Correlation, Proc Fifth Color
Imaging Conf., pages 6-9, Scottsdale, November 1997
12. E. L. Krinov, Spectral Reflectance Properties of Natural Formations, National

Research Council of Canada 1947.
13. M.J. Vrhel, R. Gershon and L.S. Iwan, Measurement and analysis of object
reflectance spectra, Color Research and Application, vol. 19, pages 4-9, 1994.
14. J. Cohen, “Dependency of the spectral reflectance curves of the Munsel color chips”
Psychon. Sci 1, pages 369-370, 1964
15. The International Commission on Illumination, http://www.cie.co.at/cie/
16. Aapo Hyvärinen, Survey on Independent Component Analysis, Helsinki University

of Technology, Laboratory of Computer and Information Science,
http://www.cis.hut.fi/~aapo/papers/NCS99web/
17. J.P.S Parkinnen, J. Hallikainen, Y. Jaaskelainen, Characteristic spectra of Munsel

colors, J. Opt. Soc.Am A/Vol. 6, No. 2/ February 1989
18. D. Freedman, R. Pisani, R. Purves, and A. Adhikari. Statistics, York, Statistics, 2rd
ed.,W.W. Norton and Co., 1991
19. Thomas H. Cormen, Charles E. Leirserson, Ronald R. Livest, Introduction to

Algorithms, Dijkstra’s Algorithm, The MIT Press, Cambridge, Massachusetts, pages
527-532, 1997
84
20. K-Nearest Neighbor Algorithm, The Machine Learning Network, http://kiew.cs.uni-
dortmund.de:8001/mlnet/instances/81d91e93-df4da3c279
21. Thomas H. Cormen, Charles E. Leirserson, Ronald R. Livest, Introduction to

Algorithms, Floyd-Warshall Algorithm, The MIT Press, Cambridge, Massachusetts,
pages 558-565, 1997
22. Forrest W. Young, Multidimensional Scaling, Kotz-Johnson (Ed.) Encyclopedia of

Statistical Sciences, Volume 5, pages 649-658, 1985
23. L.T. Maloney, Evaluation of linear models of surface spectral reflectance with small
numbers of parameters, J. Opt. Soc. Am. A, Volume 3, pages 1673-1683, 1986
85

10 1 1 3 3907

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

10 1 1 3 3907

Diunggah oleh

Hak Cipta:

Format Tersedia

LLE and Isomap Analysis of Spectra and Colour Images

B.Sc., Simon Fraser University, 1999

THESIS SUBMITTED IN PARTIAL FULFILLMENT OF

THE REQUIREMENTS FOR THE DEGREE OF

© Dejan Kulpinski 2002

SIMON FRASER UNIVERSITY

All rights reserved. This work may not be

reproduced in whole or in part, by photocopy

or other means, without the permission of the author.

Name: Dejan Kulpinski

Examining Committee: Dr. Eugenia Ternovska

Date Approved: _______________________________

Applications of LLE and Isomap methods ...........................................................................................................1

Table 1 - Symbols ............................................................................................................................................... 10

Motivation for work

Therefore, the conclusion based on the human face expression example by

Therefore, the role of LLE and Isomap techniques is examined in:

• discovering the intrinsic dimensionality of the high dimensional data

The following table of symbol definitions provides a summary of the notation

E (λ ) Spectral power distribution of ambient light

S (λ) Spectral reflectance of a surface

R(λ) Spectral Sensitivity of a sensor

RMS Root Mean Square (RMS) is a measure of the fit of the

Figure 3 - Color image and its chromaticity plots

§ the physical content of the scene,

Principal component analysis or PCA is a technique, widely used in signal

By applying SVD to X we get the following decomposition

.42 − .41 − .81 5 0 0  .71 .71 0

Variance, Residual Variance and The “Elbow” Technique

Variance is a measure of how spread out a distribution is. The variance is

The residual variance is simply

Therefore we obtain the following results:

In addition to the mathematical approach, we can use a visual approach to this

Distortion is a measure of how well a particular low-dimensional embedding

We define distortion by the following formula [1]:

Applications of LLE and Isomap

Spectra Dimensionality Estimation

The importance of color in everyday human life cannot be overstated. The

Color Histograms and Image Recognition

The Techniques Explained

The Isomap technique

1) The estimation of neighborhood graph

2) Computing the shortest path graph given neighborhood graph

3) Construction of lower dimensional embedding

In the final step of the algorithm, d-dimensional Euclidean space embedding of

The application of LLE and Isomap

A chromaticity histogram of an image is a count of the number of pixels occurring

Figure 8 - Sample of Image Library used for Chromaticity Histogram experiment.

Before generating an image’s chromaticity histogram, the image is first pre-

Table 3 - Binary Chromaticity Histogram

Table 6 – Distortion for 1050 Chromaticity Histograms based on PCA technique

To visualize the nature of the embedding we have plotted thumbnails of the

Locally Linear Embedding and Isomap techniques can be applied to image

Figure 26 - Image Grouping. Indoor vs. Outdoor

Chromaticity histograms lie in a subspace of quite low dimension, with some of

Three sets of input data[3][9] were considered:

Residual Variance in PCA is a classical methods of estimating the dimensionality of data.

Distortion measurement allows us to directly compare a dimensionality estimate obtained

For 1996 reflectance spectra the following distortion is obtained:

Table 10 - Distortion for 1996 Reflectance Spectra based on PCA technique

Table 11 – Distortion for 1996 Reflectance Spectra based on Isomap technique

Table 13 - Distortion for 170 Reflectance Spectra based on Isomap technique

R = W1* Sr B 1 + W2 Sr * B2 + W3* Sr *B3