Introduction to

Genetic Algorithms
Guest speaker:
David Hales
www.davidhales.com

GeneticAlgorithmsHistory

PioneeredbyJohnHollandinthe1970s
Gotpopularinthelate1980s
BasedonideasfromDarwinianEvolution
Canbeusedtosolveavarietyofproblems
thatarenoteasytosolveusingother
techniques

Evolutionintherealworld
Eachcellofalivingthingcontainschromosomesstringsof
DNA
EachchromosomecontainsasetofgenesblocksofDNA
Eachgenedeterminessomeaspectoftheorganism(likeeye
colour)
Acollectionofgenesissometimescalledagenotype
Acollectionofaspects(likeeyecolour)issometimescalled
aphenotype
Reproductioninvolvesrecombinationofgenesfromparents
andthensmallamountsofmutation(errors)incopying
Thefitnessofanorganismishowmuchitcanreproduce
beforeitdies
Evolutionbasedonsurvivalofthefittest

StartwithaDream

Supposeyouhaveaproblem
Youdontknowhowtosolveit
Whatcanyoudo?
Canyouuseacomputertosomehowfinda
solutionforyou?
Thiswouldbenice!Canitbedone?

Adumbsolution
Ablindgenerateandtestalgorithm:
Repeat
Generatearandompossiblesolution
Testthesolutionandseehowgooditis

Untilsolutionisgoodenough

Canweusethisdumbidea?
Sometimesyes:
ifthereareonlyafewpossiblesolutions
andyouhaveenoughtime
thensuchamethodcouldbeused

Formostproblemsno:
manypossiblesolutions
withnotimetotrythemall
sothismethodcannotbeused

Alessdumbidea(GA)
Generateasetofrandomsolutions
Repeat
Testeachsolutionintheset(rankthem)
Removesomebadsolutionsfromset
Duplicatesomegoodsolutions
makesmallchangestosomeofthem

Untilbestsolutionisgoodenough

Howdoyouencodeasolution?
Obviouslythisdependsontheproblem!
GAsoftenencodesolutionsasfixedlength
bitstrings(e.g.101110,111111,000101)
Eachbitrepresentssomeaspectofthe
proposedsolutiontotheproblem
ForGAstowork,weneedtobeableto
testanystringandgetascoreindicating
howgoodthatsolutionis

SillyExampleDrillingforOil
Imagineyouhadtodrillforoilsomewhere
alongasingle1kmdesertroad
Problem:choosethebestplaceontheroad
thatproducesthemostoilperday
Wecouldrepresenteachsolutionasa
positionontheroad
Say,awholenumberbetween[0..1000]

Wheretodrillforoil?

Solution1=300

Solution2=900

Road
0

500

1000

DiggingforOil
Thesetofallpossiblesolutions[0..1000]is
calledthesearchspaceorstatespace
Inthiscaseitsjustonenumberbutitcould
bemanynumbersorsymbols
OftenGAscodenumbersinbinary
producingabitstringrepresentingasolution
Inourexamplewechoose10bitswhichis
enoughtorepresent0..1000

Converttobinarystring
512 256 128

64

32

16

900

300

1023

InGAstheseencodedstringsaresometimescalled
genotypesorchromosomesandtheindividualbitsare
sometimescalledgenes

DrillingforOil
Solution1=300
(0100101100)

Solution2=900
(1110000100)

Road
0

OIL

1000

30
5

Location

Summary
Wehaveseenhowto:
representpossiblesolutionsasanumber
encodedanumberintoabinarystring
generateascoreforeachnumbergivenafunction
ofhowgoodeachsolutionisthisisoften
calledafitnessfunction
Oursillyoilexampleisreallyoptimisationovera
functionf(x)whereweadapttheparameterx

SearchSpace
Forasimplefunctionf(x)thesearchspaceisone
dimensional.
Butbyencodingseveralvaluesintothe
chromosomemanydimensionscanbesearched
e.g.twodimensionsf(x,y)
Searchspaceanbevisualisedasasurfaceor
fitnesslandscapeinwhichfitnessdictatesheight
Eachpossiblegenotypeisapointinthespace
AGAtriestomovethepointstobetterplaces
(higherfitness)inthethespace

Fitnesslandscapes

SearchSpace
Obviously,thenatureofthesearchspace
dictateshowaGAwillperform
Acompletelyrandomspacewouldbebad
foraGA
AlsoGAscangetstuckinlocalmaximaif
searchspacescontainlotsofthese
Generally,spacesinwhichsmall
improvementsgetclosertotheglobal
optimumaregood

Backtothe(GA)Algorithm
Generateasetofrandomsolutions
Repeat
Testeachsolutionintheset(rankthem)
Removesomebadsolutionsfromset
Duplicatesomegoodsolutions
makesmallchangestosomeofthem

Untilbestsolutionisgoodenough

AddingSexCrossover
Althoughitmayworkforsimplesearch
spacesouralgorithmisstillverysimple
Itreliesonrandommutationtofindagood
solution
Ithasbeenfoundthatbyintroducingsex
intothealgorithmbetterresultsareobtained
Thisisdonebyselectingtwoparentsduring
reproductionandcombiningtheirgenesto
produceoffspring

AddingSexCrossover
Twohighscoringparentbitstrings
(chromosomes)areselectedandwithsome
probability(crossoverrate)combined
Producingtwonewoffspring(bitstrings)
Eachoffspringmaythenbechanged
randomly(mutation)

SelectingParents
Manyschemesarepossiblesolongasbetter
scoringchromosomesmorelikelyselected
Scoreisoftentermedthefitness
RouletteWheelselectioncanbeused:
Addupthefitness'sofallchromosomes
GeneratearandomnumberRinthatrange
Selectthefirstchromosomeinthepopulation
thatwhenallpreviousfitnesssareadded
givesyouatleastthevalueR

Examplepopulation
No.
1
2
3
4
5
6
7
8

Chromosome
1010011010
1111100001
1011001100
1010000000
0000010000
1001011111
0101010101
1011100111

Fitness
1
2
3
1
3
5
1
2

RouletteWheelSelection
1
1

Rnd[0..18]=7

Rnd[0..18]=12

Chromosome4

Chromosome6

Parent1

Parent2

8
2

18

CrossoverRecombination
1011011111

1010000000

Parent1

Offspring1

1001011111

Parent2

Offspring2 1010000000

Crossover
singlepoint
random

Withsomehighprobability(crossover
rate)applycrossovertotheparents.
(typicalvaluesare0.8to0.95)

mutate

Mutation
Offspring1

1011011111

Offspring2 1010000000

Offspring1

1011001111

Offspring2 1000000000

Originaloffspring

Mutatedoffspring

Withsomesmallprobability(themutationrate)flip
eachbitintheoffspring(typicalvaluesbetween0.1
and0.001)

Backtothe(GA)Algorithm
Generateapopulationofrandomchromosomes
Repeat(eachgeneration)
Calculatefitnessofeachchromosome
Repeat
Userouletteselectiontoselectpairsofparents
Generateoffspringwithcrossoverandmutation
Untilanewpopulationhasbeenproduced

Untilbestsolutionisgoodenough

ManyVariantsofGA
Differentkindsofselection(notroulette)
Tournament
Elitism,etc.

Differentrecombination
Multipointcrossover
3waycrossoveretc.

Differentkindsofencodingotherthanbitstring
Integervalues
Orderedsetofsymbols

Differentkindsofmutation

Manyparameterstoset
AnyGAimplementationneedstodecideon
anumberofparameters:Populationsize
(N),mutationrate(m),crossoverrate(c)
Oftenthesehavetobetunedbasedon
resultsobtainednogeneraltheoryto
deducegoodvalues
Typicalvaluesmightbe:N=50,m=0.05,
c=0.9

Whydoescrossoverwork?
Alotoftheoryaboutthisandsome
controversy
HollandintroducedSchematheory
Theideaisthatcrossoverpreservesgood
bitsfromdifferentparents,combining
themtoproducebettersolutions
Agoodencodingschemewouldtherefore
trytopreservegoodbitsduringcrossover
andmutation

GeneticProgramming
Whenthechromosomeencodesanentire
programorfunctionitselfthisiscalled
geneticprogramming(GP)
Inordertomakethisworkencodingisoften
doneintheformofatreerepresentation
Crossoverentialsswapingsubtreesbetween
parents

GeneticProgramming

Itispossibletoevolvewholeprogramslikethis
butonlysmallones.Largeprogramswithcomplex
functionspresentbigproblems

Implicitfitnessfunctions
MostGAsuseexplicitandstaticfitness
function(asinouroilexample)
SomeGAs(suchasinArtificialLifeor
EvolutionaryRobotics)usedynamicand
implicitfitnessfunctionslikehowmany
obstaclesdidIavoid
Intheselatterexamplesotherchromosomes
(robots)effectthefitnessfunction

Problem
IntheTravellingSalesmanProblem(TSP)a
salesmanhastofindtheshortestdistancejourney
thatvisitsasetofcities
Assumeweknowthedistancebetweeneachcity
Thisisknowntobeahardproblemtosolve
becausethenumberofpossibleroutesisN!where
N=thenumberofcities
Thereisnosimplealgorithmthatgivesthebest
answerquickly

Problem
Designachromosomeencoding,amutation
operationandacrossoverfunctionforthe
TravellingSalesmanProblem(TSP)
AssumenumberofcitiesN=10
Afteralloperationstheproducedchromosomes
shouldalwaysrepresentvalidpossiblejourneys
(visiteachcityonceonly)
Thereisnosingleanswertothis,manydifferent
schemeshavebeenusedpreviously