Anda di halaman 1dari 58

Genetic Algorithm

Genetic Algorithms are


good at taking large,
potentially huge search
spaces and navigating
them, looking for optimal
combinations of things,
solutions you might not
otherwise find in a
lifetime.
Salvatore Mangano
Computer Design, May 1995

Structure of Biological Gen


Page 1

Genetic Algorithm

Biological Background (cell)


Every animal cell is a complex structure where many small
factories are working together
The center of this all is the cell nucleus
The nucleus contains the genetic information

Page 2

Genetic Algorithm

Biological Background (chromosomes)


The genetic information is stored in the chromosomes
Each chromosome is build of DNA (deoxyribonucleic acid).
Chromosomes in humans form are pairs.
There are 23 pairs (in the human cell).
The chromosome is divided
into parts: genes.
Genes code for properties.
The posibilities of the genes for
one property is called: allele.
Every gene has an unique position
on the chromosome: locus.

Page 3

Genetic Algorithm

Biological Background (chromosomes)

On the picture we can see a human cell with all metaphase chromosome ordered
by size and an other attribute
metaphase: the state of gen dividing when the chromosomes are situated in the plane of the equator of
cell.
Page 4

Genetic Algorithm

Biological Background (reproduction)


Reproduction of genetical information
Mitosis
Meiosis

Mitosis is copying the same


genetic information to new
offspring: there is no
exchange of information
Mitosis is the normal way of
growing of multicell structures,
like organs.
Page 5

Genetic Algorithm

Biological Background (reproduction)


Meiosis is the basis of sexual
reproduction
After meiotic division 2 gametes
appear in the process
In reproduction two gametes
conjugate to a zygote wich
will become the new individual
Hence genetic information is
shared between the parents in
order to create new offspring
Page 6

Genetic Algorithm

Biological Background (reproduction)


During reproduction errors occur
Due to these errors genetic variation exists
Most important errors are:
Recombination (cross-over)
Mutation

Page 7

Genetic Algorithm

Biological Background
(natural secection)

The origin of species: Preservation of favourable


variations and rejection of unfavourable variations.
There are more individuals born than can survive, so there
is a continuous struggle for life.
Individuals with an advantage have a greater chance for
survive: survival of the fittest.

Page 8

Genetic Algorithm

Genetic Algoritm
(in Technical Tasks)

Directed search algorithms based on the


mechanics of biological evolution.
Developed by John Holland, University of
Michigan (1970s)

Page 9

To understand the adaptive processes of natural


systems
To design artificial systems software that retains
the robustness of natural systems

Genetic Algorithm

Genetic Algoritm
(in Technical Tasks)

Provide efficient, effective techniques for


optimization and machine learning applications
Widely-used today in business, scientific and
engineering circles

Page 10

Genetic Algorithm

Classes of Search Techniques


Search techniques
Calculus-based techniques
Direct methods
Finonacci

Indirect methods
Newton

Evolutionary algorithms

Evolutionary strategies

Centralized

Simulated annealing

Dynamic programming

Genetic algorithms

Parallel

Page 11

Enumerative techniques

Guided random search techniques

Distributed

Sequential
Steady-state

Generational

Genetic Algorithm

Genetic Algoritm (Components)


A problem to solve, and ...
Encoding technique
(gene, chromosome)
Initialization procedure
(creation)
Evaluation function
(environment)
Selection of parents
(reproduction)
Genetic operators
(mutation, recombination)
Parameter settings
(practice and art)

Page 12

Genetic Algorithm

Simple Genetic Algorithm


{

_initialize population;
_evaluate population;
while Termination_Criteria_Not_Satisfied
{

_select parents for reproduction;


_perform recombination and mutation;
_evaluate population;
}
}
Page 13

Genetic Algorithm

Genetic Algoritm; Cycle of Reproduction


reproduction

selected
parents

modified
children

parents
initialization

population

deleted
members

modification

evaluated
children

evaluation

discard
Page 14

Genetic Algorithm

Cycle of Reproduction; Population


initialization

population

Elements of population (chromosomes) could be:

Bit strings
Real numbers
Permutations of element
Lists of rules
Program elements
... any data structure ...

Page 15

(0110001011010011100)
(43.2, -33.1, ... 0.0, 89.2)
(E11, E3, E7, ... E1, E15)
(R1, R2, R3, ... R22, R23)
(genetic, programming)

Genetic Algorithm

Bit string chromosome


parameters
chromosome (bit string)
a gen = 1 bit
range of integer number

linear decoding
parameter range
Page 16

x=26+13*(4/31) = 27.68
Genetic Algorithm

Bit string cromosome

Page 17

Genetic Algorithm

Real numbers chromosome


x0

x1

a gen = one real number

Real number chromosome = vector of real numbers

Page 18

Genetic Algorithm

Permutation of elements cromosome


chromosome (permutation of elements)
a gen = an order number at a given position

representation by graph

Page 19

Genetic Algorithm

Cycle of Reproduction; Reproduction


reproduction

selected
parents

parents

population
Parents are selected at random with selection
chances biased in relation to chromosome
evaluations (fitness function).

Page 20

Genetic Algorithm

Reproduction; Fitness Function


Bit string chromosome

Real numbers chromosome

Page 21

Fitness(x,y) is any calculated


value

Genetic Algorithm

Reproduction; Fitness Function


chromosome (permutation of elements)

Fitness ( cromosome ) for


example is the distance
between the points
represented positions

Page 22

Genetic Algorithm

Reproduction; Rulett Whell Selection


Cake of fitness functions
in a population
F[pv4]

F[pv5]

F[pv6]

F[pv3]

F[pv2
]
F[pv1]

F[pv7]

F[pv8]

area = fitness(x7,y7)
Page 23

Genetic Algorithm

Reproduction; Rank Selection

Ranking is a parent selection method based on


the rank of chromosomes.
Each chromosome is ranked by its fitness value.
r1 is assigned to the worst; r2 to second worst;
and so on.
Higher fitness value has the higher ranking, which
means it will be chosen with higher probability.
Calculate the sum of ranks: result is Rsum.
Parent selection: Random number generating
between 0..Rsum
Page 24

Genetic Algorithm

Reproduction; Tournament Selection

Tournament selection is one of many methods of


selection in genetic algorithms which runs a
"tournament" among a few individuals chosen at
random from the population and selects the
winner (the one with the best fitness) for
crossover.
If the tournament size is higher, weak individuals
have a smaller chance to be selected.
A 1-way tournament selection is equivalent to
random selection.
Page 25

Genetic Algorithm

Cycle of Reproduction; Modification


selected parents

modification
modified children

Modifications are stochastically triggered


Operator types are:
Crossover (recombination) (probability of crossover)
Mutation (probability of mutation)

Page 26

Genetic Algorithm

Modification; Crossover
cut

P1
P2

(0 1 1 0 1 0 0 0)
(1 1 0 1 1 0 1 0)

(1 1 0 0 1 0 0 0)
(0 1 1 1 1 0 1 0)

Ch1
Ch2

Crossover is a critical feature of genetic algorithms:


It greatly accelerates search early in evolution of
a population
It leads to effective combination of schemata
(sub solutions on different chromosomes)

Page 27

Genetic Algorithm

Modification; Mutation
random selected positions

Bit string chromosome

Before:

(1 0 1 1 0 1 1 0)

After:

(0 1 1 0 0 1 1 0)

Real numbers chromosome

Before:

(1.38 -69.4 326.44 0.1)

After:

(1.38 -67.5 326.44 0.1)

Causes movement in the search space


(local or global)
Restores lost information to the population

Page 28

Genetic Algorithm

Cycle of Reproduction; Evaluation


evaluated
children

modified
children

evaluation

The evaluator decodes a chromosome and


assigns it a fitness measure
The evaluator is the only link between a classical
GA and the problem it is solving

Page 29

Genetic Algorithm

Cycle of Reproduction; Deletion


population
discarded
members

discard

Generational GA:
entire populations (fully) replaced with each iteration
Steady-state GA:
a few members replaced each generation
Page 30

Genetic Algorithm

Cycle of Reproduction

Page 31

Genetic Algorithm

The Gene is by far the most sophisticated program around.


- Bill Gates, Business Week, June 27, 1994

Page 32

Genetic Algorithm

A Simple Example
The Travelling Salesman Problem (TSP):
We search the summa of distance between the
towns with the next conditions:
we can visit every town only ones
we have to minimize the full distance
(summa distance between the towns in a
given sequence)
Page 33

Genetic Algorithm

Representation of TSP
Representation is an ordered list of city
numbers known as an order-based GA.
1) Oulu
3) Velence
2) Budapest 4) Singapur
CityList1
CityList2

Page 34

5) Peking
6) London

7) Tokio
8) New York

(3 5 7 2 1 6 4 8)
(2 5 7 6 8 1 3 4)

Genetic Algorithm

Crossover of TSP
Crossover combines inversion and recombination:
random selected positions

Parent2

*
*
(3 5 7 2 1 6 4 8)
(2 5 7 6 8 1 3 4)

Child

(5 8 7 2 1 6 3 4)

Parent1

Page 35

Genetic Algorithm

Mutation of TSP
Mutation involves reordering of the list:
random selected positions

Before:

*
*
(5 8 7 2 1 6 3 4)

After:

(5 8 6 2 1 7 3 4)

Page 36

Genetic Algorithm

Traveling Salesman Problem

Page 37

Genetic Algorithm

Rucksack (Backpack) Packing


The problem:
We have N pieces of objects. Every object has a mass and a face value.
We have a rucksack (backpack) which has X kg weight-bearing capacity.
The task is selecting a suitable subset of the objects, where the face
value is maximal and the sum mass of objects are limited to X kg.
We solve the problem applying the genetic algoritm with bitstring
chromosome. The coding in the chromosome gives whether the object is
selected or not.
The upper matrix containing the parent chromosomes the lower the
crossovered and mutated chromosomes. The numbers beside the
matrixes are the sum of face values of selected objects with other words
the fitness function.
Page 38

Genetic Algorithm

Rucksack (Backpack) Packing

Page 39

Genetic Algorithm

Maximum Value of Function


Finding the maximum value of a function
(MVFN):
We finding the maximal value of an explicit function,
(the global maximum):

the input parameter ranges are given


for the search

Page 40

Genetic Algorithm

Representation of MVFN
The chromosome is coded with a vector of real
numbers.
x = { -20.0, +12.0 }
y = { -30.0, +100.0 }

range of search x
range of search y

c(i) = [ x(i), y(i) ]

the form of chromosome

c(1) = [ 3.234, 5.111 ]


c(2) = [ 10.12, -3.9234 ]

chromosome(1)
chromosome(2)

Page 41

Genetic Algorithm

Crossover of MVFN
Parent(1)= [x(1), y(1)];

Parent(2) = [x(2), y(2)];

a = random number between {0..1}


Child(1) = [ a*x(1) +
a*y(1) +
Child(2) = [ {1-a}*x(1)
{1-a}*y(1)
Page 42

{1-a}*x(2),
{1-a}*y(2) ]
+ a*x(2),
+ a*y(2) ]

/ type!!! /

/ ranges!!! /
/ ranges!!! /
Genetic Algorithm

Mutation of MVFN
The mutation a randomly selected element value
changing randomly:
a,b = random numbers with range {0..1} / type!!! /
c = constant { for example = 0.25 }
Before: [ x1, y1 ]
After: [ x1 + (x1max - x1min)*c*a,
y1 + (y1max - y1min)*c*a ]
Page 43

/ ranges!!! /
/ ranges!!! /
Genetic Algorithm

Type of Random Numbers:


Uniform Distribution

f ( x ) = (b a )
0

Page 44

if a x b
otherwise

Genetic Algorithm

Type of Random Numbers:


Exponential Distribution

e x
f (x) =
0

Page 45

for 0 x
otherwise

Genetic Algorithm

Type of Random Numbers:


Normal Distribution

y
f ( x ) =
e
2
( x ) 2
y=
2 2

Page 46

Genetic Algorithm

Type of Random Numbers:


Triangular Distribution
checkpoint = ( usual -minimum ) /
( maximum - minimum );

triangledistribution = ( random <=


checkpoint )?
sqrt ( random * ( maximum - minimum
) * (usual - minimum) )+minimum :
maximum - sqrt ((1random)*(minimummaximum)*(usual-maximum)) ;

Page 47

Genetic Algorithm

Maximum Value of Function


The function:
f(x, y) = 20 - (x1 - x0)^2 + (y1 - y0)^2
where x0 and y0 are constants

We can find also the minimum


value, in this case we
maximazing the - f(x,y) function.
Page 48

Genetic Algorithm

Maximum Value of Function


2

f ( x ) = 20 x[ i ] 5.0
i =1

Page 49

Genetic Algorithm

Maximum Value of Function


f ( x ) = 20 ( x[ i ] x[ i0 ] )
N

i =1

Page 50

Genetic Algorithm

Search the Structure of a Function


We have big number of measuring data where the
input parameter(s) is(are) known to the measured
data.
For example the measuring data is the area of a circle
and the input parameter the radius of the circle.
We want to find a suitable structure of function
is giving the connection between the input parameter
and measured data.
Page 51

Genetic Algorithm

Search the Structure of a Function

Page 52

Genetic Algorithm

Issues for GA Practitioners

Choosing basic implementation issues:

representation of information
population size, mutation rate, ...
selection, deletion policies
crossover, mutation operators

Termination Criteria
Performance, scalability
Solution is only as good as the evaluation
function (often hardest part)

Page 53

Genetic Algorithm

Benefits of Genetic Algorithms

Concept is easy to understand


Modular, separate from application
Supports multi-objective optimization
Good for noisy environments
Always have an answer; answer became
better and better with time
Inherently parallel; easily distributed

Page 54

Genetic Algorithm

Benefits of Genetic Algorithms

Many ways to speed up and improve a GAbased application as knowledge about problem
domain is gained.
Easy to exploit previous or alternate solutions.
Flexible building blocks for hybrid applications.
Substantial history and range of use.

Page 55

Genetic Algorithm

When to Use a GA

Alternate solutions are too slow or overly


complicated
Need an exploratory tool to examine new
approaches
Problem is similar to one that has already been
successfully solved by using a GA
Want to hybridize with an existing solution
Benefits of the GA technology meet key problem
requirements

Page 56

Genetic Algorithm

Some GA Application Types


Domain

Application Type

Control

gas, pipeline, pole balancing, missile evasion, pursuit

Design
Scheduling

semiconductor layout, aircraft design, keyboard


configuration, communication networks
manufacturing, facility scheduling, resource allocation

Robotics

trajectory planning

Machine Learning
Signal Processing

designing neural networks, improving classification


algorithms, classifier systems
filter design

Game Playing

poker, checkers, prisoners dilemma

Combinatorical
Optimization

set covering, traveling salesman, routing, bin packing,


graph coloring and partitioning

Page 57

Genetic Algorithm

Thank you for your attention


Questions?

Page 58

Genetic Algorithm