Genetic Algorithms

Introduction
Real world optimization problems are complex,
have multiple conflicting objectives, etc.
Large search or design space
Difficult to obtain global optimum in a reasonable
time.
Evolutionary algorithms (EA) can be used.
Genetic Algorithms (GA) make use of EA
approach.
GA uses principle of Darwinian evolution theory i.e.,
survival of the fittest.
GA operates mainly on a population search basis.
Contd
Starts from a population of possible solutions
(called individuals or chromosome) and move
towards the optimal one.
Phenotype: Objects forming possible solution sets
to the original problem.
Genotype: The encoding (representation) of the
individuals.
In GA, the variables are represented as strings of
numbers (normally binary).
Contd
If there are 'n' variables and encoding provides a
string of length 'l' to each variable, then each
solution vector will have a total string length of 'nl'.
Example:
Number of variables = 3 (x1, x2, and x3)
Encoding provides the string length of 4 for each variable.
Then the length of each chromosome is 12.
If x1 = 7, x2 = 4, and x3 = 1, then chromosome will be
0 1 1 1 0 1 0 0 0 0 0 1
x1 x2 x3
Contd
Each chromosome has an associated value known
as fitness (i.e. the quality).
Derived using a fitness function.
Forms the basis of the selection process.
GAs update population iteratively.
By evaluating chromosomes using a fitness function and
keeping only the fitter ones in a new generation.
Some of these individuals are admitted unchanged.
Others are subjected to genetic operators such as
crossover and mutation to create new offspring.
 n is the number of
Pseudocode individuals in the population;
is the fraction of the
Algorithm: GA(n, , ) population to be replaced by
1. // Initialise generation 0: crossover in each iteration;
2. k := 0; and is the mutation rate.
3. Pk := a population of n randomly-generated individuals;
4. // Evaluate Pk:
5. Compute fitness(i) for each i Pk;
6. do
7. { // Create generation k + 1:
8. // 1. Copy: (elitist selection) Elitism
9. Select (1 ) n members of Pk and insert
into Pk+1;
Contd
10. // 2. Crossover:
11. Select n members of Pk; pair them up;
produce offspring; insert the offspring into Pk+1;
12. // 3. Mutate:
13. Select n members of Pk+1; invert a
randomly-selected bit in each;
14. // Evaluate Pk+1:
15. Compute fitness(i) for each i Pk+1;
16. // Increment:
17. k := k + 1;
18. }
19. while fitness of fittest individual in Pk is not high enough;
20. return the fittest individual from Pk;
Representation (Encoding)
One of the biggest challenge in GA.
Assigning a letter (or a number) to each of the
simplest element in a solution.
As number of elements in the solution is finite, and
hence each is represented by a string of finite
length (known as a chromosome).
The easiest representation is a bit representation {0,
1} as it simplifies crossover and mutation.
However, depending upon a problem other
representations can also be determined and used.
Example
Find optimal quantity of three ingredients in a recipe.
Use integers {1, 2, 3, ..., 9} denoting the number of table spoons of
each ingredient.
Some possible solutions are 1-1-1, 2-1-4, 3-3-1, etc.
The traveling salesperson problem to traverse 4 cities.
Use integers or alphabets to represent cities.
A solution is any permutation of cities.
Amritsar (A or 1)
Amritsar (A or 1)
Ludhiana (L or 3)
Ludhiana (L or 3)

Jalandhar (J or 2)
Patiala (P or 4) Jalandhar (J or 2)
Patiala (P or 4)
1-4-2-3 (A P J L) 1-2-3-4 (A J L P)
Contd Color Shape Size Object
Red (100) Round (10) Small (10) Apple (1000)
Rule-based Green (010) Square (01) Big (01) Orange (0100)
system. Blue (001) Banana (0010)
Pear (0001)
Each rule is a 11-bit string.
If color = Red, shape = Round, and size = Small then object =
Apple. (100 10 10 1000)
Timetabling Number Faculty Room Time
Chromosomes consists of various 1 Prof. XYZ E202 8:00
triplets '123' means 'Prof. XYZ 2 Mr. ABC F103 9:00
teaches at 10:00 in F103'. 3 PQR G301 10:00
Faculty Room Time

Chromosomes
Contd
A representation for the fire-station location
problem.

If 1 represents a fire station, then chromosome is a

14-bit string, like 1 0 1 0 1 0 0 0 0 1 0 0 1 0.
Fitness Function
Used to score and rank the individuals
(chromosomes).
Task specific, non-monotonic, and usually give a real
number to a chromosome.
Choosing the right fitness function is very important,
but also quite difficult
Example:
For binary numbers, compute an equivalent integer value.
For traveling salesperson problem, length of the route.
For classification problems, the percent of correct
classifications on a given set.
Selection
GA works on Darwin's evolution theory.
Survival of the fittest.
Select the best ones to create new offspring for the
next generation.
There are many selection methods.
Roulette wheel selection,
Boltzmann selection,
Tournament selection,
Rank selection,
Steady state selection, etc.
Roulette Wheel Selection
Chromosomes are selected according to their
fitness.
Each individual is selected with a probability
proportional to its fitness value.
The probability that an individual i is selected, Pi, is
computed as fitness (i )
Pi n

fitness ( j )
j 1

Weak solutions are eliminated and strong solutions

survive to form the next generation.
Example
Consider a population,
Chromosome Fitness Value % of Total Fitness C4: 17.5%
1011 0110 1101 1001 125 31.25 C1: 31.25%
0101 0011 1110 1101 95 23.75 C3: 27.5%
0001 0001 1111 1011 110 27.5 C2: 23.75%
1011 1111 1011 1100 70 17.5

Plot a pie chart, randomly generate four numbers

between 1 and 100.
Likeliness that the numbers generated would fall in the
region of a chromosome is directly related to the region
that chromosome is covering.
Rank Selection
Sorts or ranks chromosomes by fitness.
Chromosome Fitness Value Rank
1011 0110 1101 1001 125 4
0101 0011 1110 1101 95 2
0001 0001 1111 1011 110 3
1011 1111 1011 1100 70 1
Chromosome with the highest rank is more likely to
be selected than those having lower rank in the
sorted list.
Slower convergence rate.
Tournament Selection

Randomly select m
Keep the fittest one
chromosomes

NO Population
filled?

Old Population Fitness YES

Chromosome 1 F1
Chromosome 2 F2 Continue with other
operations to get the
next population.
Chromosome N FN
Reproduction Methods
Mutation
Randomly change one or more digits in a chromosome.
How often to do mutation, how many digits to change
are adjustable parameters.
Crossover:
In crossover, portions of two parents from the current
generation are combined to create two offspring for the
next generation.
k-point Crossover:
One-point, and two-point crossovers are the simplest and most
widely applied crossover methods.
Crossover sites is selected at random over the string length.
Crossover

1 0 0 1 0 1
Mutation
Mutation helps to add diversity to the population. It
works as a random experimentation.
Mutation can help the population to avoid local
maximum.
Example:
1. 10111011 2. 11101001000
10111111 11001011000
Example
Consider a GA with chromosomes consisting of six genes xi =
abcdef, and each gene is a number between 0 and 9. Let there be
four chromosomes in the population:

x1 = 4 3 5 2 1 6 x2 = 1 7 3 9 6 5
x3 = 2 4 8 0 1 2 x4 = 9 0 8 1 2 3

and the fitness function be f(x) = (a + c + e) (b + d + f).

1. Sort the chromosomes by their fitness.

2. Do one-point crossover in the middle between the 1st and 2nd
fittest, and two-points crossover (points 2, 4) for the 2nd and 3rd
fittest.
3. Calculate the fitness of all the offspring.
Sort the chromosomes by their fitness
x = abcdef
Fitness function:
f(x) = (a + c + e) (b + d + f):
f(x1) = f(4 3 5 2 1 6) = (4 + 5 + 1) (3 + 2 + 6) = -1
f(x2) = f(1 7 3 9 6 5) = (1 + 3 + 6) (7 + 9 + 5) = -11
f(x3) = f(2 4 8 0 1 2) = (2 + 8 + 1) (4 + 0 + 2) = 5
f(x4) = f(9 0 8 1 2 3) = (9 + 8 + 2) (0 + 1 + 3) = 15

Sorted order of chromosomes on the basis of fitness

values is x4, x3, x1 and x2.
Crossover
One-point crossover in the middle between the 1st
and 2nd fittest, i.e. x4 and x3:
x4 = 9 0 8 1 2 3 O1 = 9 0 8 0 1 2
x3 = 2 4 8 0 1 2 O2 = 2 4 8 1 2 3
Two-points crossover (points 2, 4) for the 2nd and 3rd
fittest, i.e. x3 and x1:
x3 = 2 4 8 0 1 2 O3 = 2 4 5 2 1 2
x1 = 4 3 5 2 1 6 O4 = 4 3 8 0 1 6
Calculate fitness of all the offspring
x = abcdef
Fitness function:
f(x) = (a + c + e) (b + d + f):

f(O1) = f(9 0 8 0 1 2) = (9 + 8 + 1) (0 + 0 + 2) = 16
f(O2) = f(2 4 8 1 2 3) = (2 + 8 + 2) (4 + 1 + 3) = 4
f(O3) = f(2 4 5 2 1 2) = (2 + 5 + 1) (4 + 2 + 2) = 0
f(O4) = f(4 3 8 0 1 6) = (4 + 8 + 1) (3 + 0 + 6) = 4
Summary
GAs are easy to apply to a wide range of problems.
Optimization problems like the traveling
salesperson problem, to inductive concept
learning, scheduling, and layout problems.
Results can be very good on some problems, and
rather poor on others.
Only mutation slows down the algorithm.
Crossover makes the algorithm significantly faster.
 0/1 Knapsack

Using Genetic Algorithm

Problem Statement
Choose a subset from N items such that the
corresponding profit sum is maximized without
having the weight sum to exceed the capacity C.
N
Maximize px
i 1
i i
wi and pi are the weight
and profit of ith item.
N xi = 1, if item i is
Subject to w x
i 1
i i C included in the knapsack,
otherwise xi = 0.
where xi {0,1}, i 1,2,...N
Example 1 Name Weight Value
A 45 3
Chromosome is a 4-bit string. B 40 5
{xA xB xC xD} C 50 8
D 90 10
Population size = 4. C = 100.
First two fittest chromosomes selected as it is.
3rd and 4th fittest use for one-point crossover in the
middle followed by single bit mutation of first
offspring. Bits chosen for mutation follows this
cyclic order (xD, xC, xB, xA).
Initial population: 1 1 1 1, 1 0 0 0, 1 0 1 0, 1 0 0 1.
Output the result after 12 iterations.
 Name Weight Value
Solution C = 100 A 45 3
B 40 5
Population Fitness Rank Operations C 50 8
0 1111 0 3 D 90 10
1000 3 2 1 1 1 1 1 1 0 1 1 1 0 0

1010 11 1 1 0 0 1 1 0 1 1 1 0 1 1
1001 0 4
1 1010 11 1
1000 3 3 1 0 0 0 1 0 1 1 1 0 0 1

1100 8 2 1 0 1 1 1 0 0 0 1 0 0 0
1011 0 4
2 1010 11 1
1100 8 2 1 0 0 0 1 0 0 1 1 1 0 1

1001 0 4 1 0 0 1 1 0 0 0 1 0 0 0
1000 3 3
3 1010 11 1
1100 8 2 1 0 0 0 1 0 0 1 0 0 0 1

1101 0 4 1 1 0 1 1 1 0 0 1 1 0 0
1000 3 3
 Name Weight Value
Contd C = 100 A 45 3
B 40 5
Population Fitness Rank Operations C 50 8
4 1010 11 1 D 90 10
1100 8 3 1 1 0 0 1 1 0 0 1 1 0 1

0001 10 2 1 1 0 0 1 1 0 0 1 1 0 0
1100 8 4
5 1010 11 1
0001 10 2 1 1 0 0 1 1 0 1 1 1 1 1

1101 0 4 1 1 0 1 1 1 0 0 1 1 0 0
1100 8 3
6 1010 11 1
0001 10 2 1 1 0 0 1 1 1 1 1 0 1 1

1111 0 4 1 1 1 1 1 1 0 0 1 1 0 0
1100 8 3
7 1010 11 1
0001 10 2 1 1 0 0 1 1 1 1 0 1 1 1

1011 0 4 1 0 1 1 1 0 0 0 1 0 0 0
1100 8 3
 Name Weight Value
Contd C = 100 A 45 3
B 40 5
Population Fitness Rank Operations C 50 8
8 1010 11 1 D 90 10
0001 10 2 1 0 0 0 1 0 1 1 1 0 1 0

0111 0 4 0 1 1 1 0 1 0 0 0 1 0 0
1000 3 3
9 1010 11 1
0001 10 3 0 0 0 1 0 0 0 0 0 0 1 0

1010 11 2 0 1 0 0 0 1 0 1 0 1 0 1
0100 5 4
10 1010 11 1
1010 11 2 0 0 1 0 0 0 0 1 0 1 0 1

0010 8 3 0 1 0 1 0 1 1 0 0 1 1 0
0101 0 4
11 1010 11 2
1010 11 3 1 0 1 0 1 0 0 1 0 0 0 1

0101 0 4 0 1 0 1 0 1 1 0 0 1 1 0
0110 13 1
Contd
Name Weight Value C = 100
A 45 3 Population Fitness Rank
B 40 5 12 0110 13 1
C 50 8 1010 11 3
D 90 10 0001 10 4
0110 13 2
Output the fittest individual, i.e. 0110.
Total profit = 0 3 + 1 5 + 1 8 + 0 10 = 13.
Total weight = 0 45 + 1 40 + 1 50 + 0 90
= 90 100.