ImprovingtheAccuracyof

RemoteSensingClassifications
Mahesh Pal
Department of Civil Engineering
N. I. T. Kurukshetra
Haryana India

 Classification system used

Neural Network
Decision tree

Ensemble Methods used

Boosting
Bagging

Data used
Results

Artificial Neural Network

three layer backpropagation neural network.

First layer, called Input layer, defines
hyperplane decision boundaries.
Second layer, called hidden layer, intersects
hyperplanes decision boundaries to define
polygonal regions.
Third layer , called output layer, provides the
union of these regions into various classes.
The each element in a layer is known
neuron.
The neurons in every layer are fully
connected to the nurons in proceeding layer.
Training samples are assigned to the input
layer.
Output class is defined during the training
process and assigned to the output layer.
Connection weights are adjusted during
training such that the error between the
actual output of the network and desired
output are minimised

Decision Tree Classifier

A multistage or hierarchical classifier

(break up of complex decision into a
union of several simpler decision)
-using different subset of features at
various decision levels.
A tree is composed of a root node
(containing all the data), a set of
internal nodes (splits), and a set of
terminal nodes (leaves). Each node in
a decision tree has only one parent
node and two or more descendent
node.
Elliptical nodes are decision nodes
whose two descendants are
determined by a threshold on a
specified feature value
The square nodes are terminal nodes
and are assigned a class label.

Intuition behind Bagging and Boosting

To diagnose a Patient based on its symptoms
Ask several doctors in place of one.
Certain diagnosis occurs more than the others
choose this a the final diagnosis -------

Bagging

Assign weights to the values of each doctors Diagnosis

weight criteria -- previous diagnosis/experience of the doctors
Reach at a final diagnosis

----

Boosting

Bagging (Breiman, 1996)

Same base classifier is used
Bootstrap samples are used (randomlydrawnsamples,with
replacement,Nexamples,whereNisthesizeoftheoriginal
trainingset).
Learningsystemgeneratesaclassifierfromthesample.
Aggregatesalltheclassifiersgeneratedfromthedifferenttrial
toformthefinalclassifier.
Toclassifyaninstance,everyclassifierrecordsavoteforthe
classtowhichitbelongs,andtheinstanceislabelledasa
memberoftheclasswiththemostvotes.
Ifmorethanoneclassjointlyreceivesthemaximumnumberof
votesthanwinnerisselectedusingrandomselection.

Boosting
A method using same training data with different weights to
produce a series of classifiers.
Boostingassignsaweighttoeachobservationthehigherthe
weight,themorethatobservationinfluencestheclassifier
Ateachtrial,thevectorofweightsisadjustedandtheweight
ofmisclassifiedobservationsisincreased.
The final classifier aggregates the classifiers generated after
eachiterationbyvoting.
Eachclassifiersvoteisafunctionofitsaccuracy.
For this study AdaboostM1 (Freund and Schapire, 1996) is
used

Data used
ETM+ data near a town LittleportinUK,2000.
Classificationprobleminvolvessevenlandcovertypes
(wheat,potato,sugarbeet,onion,peas,lettuceandbeans).
Referenceimagewascreatedusingfieldsurvey.
Atotalof4737pixelsarerandomlyselected.
Totalpixelsweredividedintwoparts2700fortraining
and2037fortesting.

Classification methods
A univariate decision tree classifier with gainratioasattribute
selectionmeasuresanderrorbasedpruningwasused.
ABackpropagationneuralnetworkwithonehiddenlayer
havingtwentysixnodeswasused.
Studieswithavalidationdatasetsuggeststhat2200iteration
withlearningrateof0.25providesgoodresults.
10iterationofbaggingandboostingwascarriedout.
Totalaccuracyandkappacoefficientwerecalculatedusing
confusionmatrices.
Trainingtimeofbothclassifierswithbaggingandboosting
alsorecorded.

Results
Classification accuracies
Data set ETM+

Decision Tree
Accuracy Kappa
(%)
value

Without
bagging/
boosting

83.8

Neural Network
Accuracy Kappa
(%)
value

0.811 87.78 0.86

Boosting

87.43 0.853 88.10

0.86

Bagging

87.28 0.852 90.23

0.89

Training time
Classifier

Without Boosting
boosting/
bagging

Bagging

Neural
network

975.4
seconds

10846.1
seconds

10200.9
seconds

Decision
tree

0.53
seconds

17.02
seconds

12.4
seconds

Conclusions
Asmallpercentageincreaseinaccuracyisdifficultto
generatewhentheoverallclassificationaccuracylevel
exceeds80%,so
DT perform very well with both boosting and bagging,
improving accuracy by about 4%.
An improvement of about 2.5% in classification accuracy
with NN is good as these classifier are more stable than DT.
Training time with NN is quite large in comparison with
DT classifier.
NNperformwellwithbaggingincomparisontoboosting.