Kaveh Amouzgar
Kaveh Amouzgar
This thesis work has been carried out at the School of Engineering in
Jönköping in the subject area Product Development and Materials Engineering.
The work is a part of the master’s degree.
The authors take full responsibility for opinions, conclusions and findings
presented.
Date: 2012-05-30
In this thesis, the basic principles and concepts of single and multi-objective Ge-
netic Algorithms (GA) are reviewed. Two algorithms, one for single objective and
the other for multi-objective problems, which are believed to be more efficient,
are described in details. The algorithms are coded with MATLAB and applied
on several test functions. The results are compared with the existing solutions
in literatures and shows promising results. Obtained pareto-fronts are exactly
similar to the true pareto-fronts with a good spread of solution throughout the
optimal region. Constraint handling techniques are studied and applied in the two
algorithms. Constrained benchmarks are optimized and the outcomes show the
ability of algorithm in maintaining solutions in the entire pareto-optimal region.
In the end, a hybrid method based on the combination of the two algorithms is
introduced and the performance is discussed. It is concluded that no significant
strength is observed within the approach and more research is required on this
topic. For further investigation on the performance of the proposed techniques,
implementation on real-world engineering applications are recommended.
Keywords
1 Introduction 1
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.3 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.4 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2 Theoretical background 3
i
3 Implementation 24
4 Test Results 36
4.2 Multi-Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5 Conclusion 54
6 Bibliography 55
ii
List of Figures
1 Trade-off curve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 Min-Min pareto-front . . . . . . . . . . . . . . . . . . . . . . . . . . 13
iii
25 Pareto-front of OSY test function . . . . . . . . . . . . . . . . . . . 51
List of Tables
iv
1 Introduction
1.1 Background
Previous works by Beasley and Bull (1993); Coello (2007); Deb (1995, 2001, 2002,
2004); Deb et al. (2001); Fonseca and Fleming (1993); Haupt et al. (2004); Kim
et al. (2004); Kukkonen (2006); Man et al. (1996); Zitzler and Thiele (1998);
Zitzler et al. (2001) on the theory, concepts and algorithms of single and multi-
objective optimization using evolutionary algorithms.
Previous studies by Deb (2000); Deb et al. (2002); Jimnez et al. (1999); Kuri-
Morales and A.Gutierrez-Garcia (2002); Mezura-Montes et al. (2003); T. Ray
(2001) on constraint handling methods.
Test functions and their comparison has been studied by (Binh and Korn, 1997);
Deb (1991, 1999); Gamot and Mesa (2008); Kuri-Morales and A.Gutierrez-Garcia
(2002); Kursawe (1991); Osyczka and Kundu (1995); Srinivas and Deb (1994);
Tanaka and Watanabe (1995) and Zitzler et al. (2000).
1
1.3 Delimitations
Genetic Algorithm is the only method used in developing the technique. Other
evolutionary methods like Evolution strategies, Evolutionary programming and
Genetic Programming are not considered in the thesis.
1.4 Outline
Section 4: The results obtained from the benchmarks are illustrated and com-
pared with references.
2
2 Theoretical background
When only one objective function involves in the problem, it is called single-
objective optimization, however in most real world problems more than one objec-
tive function is required to be optimized, and therefore these problems are named
multi-objective optimization.
Deb (2001) classified optimization solving methods into following two major cat-
egories:
• Classical methods
• Evolutionary methods
The classical methods commonly use a single random solution, updated in every
iteration by a deterministic procedure to find the optimal solution. These methods
are classified into two distinct groups: direct methods where only the objective
function and the constraints value are used to find the optimum and gradient-
based methods whereas the first and second derivative of objective function and/or
constraints are applied to find the search direction and optimal solution (Deb,
2001).
3
2.2 Single-Objective Optimization
This method was inspired by the evolutionary process of human being and the
interests for imitating living being is increasing since 19600 s. Evolutionary method
mimics the evolution principle of nature which results in a stochastic search and
optimization algorithm. It also can out pace the classical method in many ways
(Gen and Cheng, 1997).
Gen and Cheng (1997) divides the method into three main types: genetic al-
gorithm, evolutionary programming and evolutionary strategy while Deb (2001)
describes an additional type to the three above-mentioned: genetic programming.
Genetic algorithm was first introduced by John Holland (1975) in 19700 s, whereas
one of his students David Goldberg had an important contribution in popularizing
this method in his dissertation by solving a complex problem (Haupt et al., 2004).
4
fit individual (parents) has a potential to have a better fitness compared to both
parents called super-fit offspring. By this principle the initial population evolves
to a better suited population to their environment in each generation (Beasley and
Bull, 1993).
After representation a fitness or objective function is required. Also during the run
a selection of parents for reproduction and recombination for creating offspring is
essential. These aspects are called GA0 s operators (Beasley and Bull, 1993).
They are number of different crossover operators in literature but the main concept
is selecting two strings of solution (chromosomes) from the mating pool of selection
operator and exchanging some portion of these two strings from a random selected
point(s). Single point cross over is one basic type of this operator for binary GA
(Deb, 2001).
5
population and increase the possibility of not losing any potential solution and find
the global optimal, while cross over operator is a technique of rapid exploration
of search space (Beasley and Bull, 1993).
To sum up, the selection operator selects and maintains the good solutions; while
crossover recombines the fit solutions to create a fitter offspring and mutation
operator randomly alter a gene or genes in a string to hopefully find a better
string (Deb, 2001).
There are some difficulties in binary-coded GAs, including inability to solve the
problems where the values of variables have continuous search space or when the
required precision is high. According to Deb (2001) hamming cliffs related to
certain strings (01111 or 11110) is one of the difficulties where altering to a near
neighbour string requires changes in many genes. He also claims necessity of large
strings (chromosomes with many genes) in order to fulfil a necessary precision
which in result increases the size of population, as another struggle for binary
GAs. Therefore using floating point numbers to represent the variables in most
problems is more logical which requires less storage than binary coded strings. In
addition, since there is no need for decoding the chromosomes before evaluation
of objective function in selection phase the real parameter GA (in some literature
called continuous GA) is inherently faster than binary GA (Haupt et al., 2004).
Since the real value of parameters are directly used to find the fitness value in
selection operator and there is no decoding to a string in real parameter GAs, this
operator does not alter with binary GA selection operators and the same operators
can be used in real parameter GAs. On the other hand, since the cross over and
mutation operators used in binary GAs are based on strings and alteration in
genes (bits), new cross over and mutation operators shall be defined for this type
of GA.
Deb (2001) outlines some real parameter crossover operators such as linear crossover,
naive crossover, blend crossover (BLX), simulated binary crossover (SBX), fuzzy
recombination operator, unimodal normally distributed crossover (UNDX), sim-
plex crossover (SPX), fuzzy connectives based crossover and unfair average crossover.
Other cross over operators including parent centric crossover (PCX), modified
PCX (mPCX) are recommended in literatures (Deb and Joshi, 2001).
6
Since in real parameter crossover operator two or more parents directly recombine
to create on or more offspring and it has the same concept as mutation operator,
a question comes up: Is there a good reason for using a mutation operator along
with crossover operator? The debate still remains, however Deb (2001) argues, the
different between these two operators is in the number of parent solutions selected
for perturbation. He claims if offspring is created from one parent the operator
is mutation while offspring created from more than one parent is crossover. He
also mentions some common mutation operators in his book: Random mutation,
non-uniform mutation, normally distributed mutation and polynomial mutation.
Step 1 (SP): From the population B (set B) of size N, the best parent and µ − 1
other parents are randomly selected. These µ solutions create Set P.
Step 2 (GP): λ offspring are created from µ parents in set P, with using any
recombination operator, which creates Set C.
Step 3 (RP): r random parents are chosen from set B, which creates Set R.
Step 4 (UP): r random parent chosen in step 3 (RP) are replaced with r best
solutions from the combined set C ∪ R (set RC), in set B.
7
Several parametric studies such as Deb (2001, 2002, 2004); Kita (2001), compare
the performance of G3 model to other evolutionary algorithms, and in all of the
studies G3 model has shown a better performance and robustness. Also using
different recombination operators has been examined and the overall result shows
faster computation time and lower number of evaluation required to meet a desired
accuracy of a parent centric recombination operator (PCX) proposed by Deb et al.
(2001), which will be briefly described.
Deb et al. (2001) suggests a variation operator (combination of crossover and mu-
tation operator) for this algorithm, called parent centric recombination operator
(PCX). A parent centric operator ensures identically of population mean of the
total offspring population to that of the parent population while mean centric
operators preserve the mean between the participating parents and resulting off-
spring. The paper states the benefit of parent centric recombination operators
over mean centric operator, as the parents are selected from the fittest solution
in selection plan and in most real parameter optimization problems it is assumed
that the solutions near the parents can be the potential good solutions. Therefore,
creating new solutions close to parents as how it is in PCX is a steady and reliable
search technique.
The mean vector ~g of the chosen parents is computed. For each offspring, one
parent ~x(p) is chosen with equal probability. The direction vector d~ = ~g − ~x(p) is
calculated. Thereafter from each of the µ − 1 parents perpendicular distance Di to
the line d~(p) are computed and their average D̄ is found. The offspring is created
as follows: µ
X
(p)
~y = ~x ~(p)
+ wζ d + wη D̄~e(i)
i=1,i6=p
where ~e(i) are the µ − 1 orthonormal bases that span the subspace perpendicular
to d~(p) . Parameters wζ and wη are zero-mean normally distributed variables with
variance wζ2 and wη2 , respectively.
8
2.2.7 Constraint Handling
Most existing constraint handling methods in literatures are classified in five cat-
egories which Deb (2001) describes them briefly:
• Hybrid methods.
In the thesis, the method based on penalty function is used for single-objective
optimization.
• Death Penalty
• Static Penalties
• Dynamic Penalties
• Annealing Penalties
• Adaptive Penalties
• Segregated GA
In the Static Penalty method which is implemented in this section, the penalty
parameters do not change within generations and is only applied to infeasible
solutions.
There are number of approaches in this method suggested by authors but Morales
et al. (1997) penalizes the objective function of infeasible solutions by using the
information on the number of violated constraints. His approach is formulated as
follows:
9
(
f (x), if xisf easible,
F (x) = Ps K
K − i=1 m , otherwise.
In real world applications, most of the optimization problems involve more than
one objective to be optimized. The objectives in most of engineering problems are
often conflicting, i.e., maximize performance, minimize cost, maximize reliability,
etc. In the case, one extreme solution would not satisfy both objective functions
and the optimal solution of one objective will not necessary be the best solution for
other objective(s). Therefore different solutions will produce trade-offs between
different objectives and a set of solutions is required to represent the optimal
solutions of all objectives.
Figure 1 shows the trade-off curve of decision making involved in buying a house
problem.
The trade-off curve reveals that considering the extreme optimal of one objective
(price) requires a compromise in other objective (house area). However there
exists number of trade-off solutions between the two extreme optimal, that each
are better with regards to one objective.
The multi-objective optimization problems may also have one or more constraints
10
Figure 1: Trade-off solutions illustrated for a house-buying decision-making
11
space corresponds to a point in objective space.
• Feasible and infeasible solutions: A solution that satisfies all the con-
straints (inequality and equality) and variable bounds is referred to as a
feasible solution. On the other hand, a solution that does not satisfy all
constraints and variable bounds is called an infeasible solution.
• Utopian objective vector: A vector that all of its components are marginally
smaller (in case of minimization MOP) than that of the ideal objective vector
is called utopian objective vector. In other words:
• Linear and non- linear MOOP: If all objectives and constraints are linear
the problem is named a linear optimization problem (MOLP). In contrast, if
one or more of the objectives and/or constrains are non-linear the problem
in non-linear MOOP (Deb, 2001).
12
non-convex part of the trade-off curve. Moreover many of the existing algo-
rithms can only be used for convex problems. Convexity can be defined on
both of spaces (objective and decision variable space). A problem can have
a convex objective space while the decision variable space is non-convex.
1. The solution x(1) is no worse than x(2) with respect to all objectives
value, or fj (x(1) ) 7 fj (x(2) ) for all j = 1, 2, ..., M .
2. The solution x(1) is strictly better than x(2) in at least one objective
value, or fj̄ (x(1) ) C fj̄ (x(2) ) for at least one j̄ ∈ {1, 2, ..., M }.
P ∗ = {x ∈ Ω | ¬∃ x0 ∈ Ω F (x0 ) F (x)}.
13
Figure 2: Pareto-front of a Min–Min problem
They are several methods and algorithms towards finding the non-dominated set
of solutions from a given population in an optimization problem. Deb (2001)
describes three of the most common methods in his book from a naive and slow
to an efficient and fast approach.
Approach 3 has the least computational complexity among the three and according
to Kung and Luccio (1975) is the most efficient method. In all methods the concept
of domination is used to compare the solution with respect to different objective
functions.
14
2.3.4 Approaches Towards Multi-Objective Optimization
15
1. To find a set of non-dominated solutions with the least distance to pareto-
optimal set.
Recall from section 2.1, that classifies optimization solving methods into; classi-
cal and evolutionary method , the classification is also valid for multi-objective
optimization problems.
Requirement of little prior knowledge from the problem , less vulnerability to shape
and continuity of pareto-front, easy implementation, robustness and the ability to
be carried out in parallel are some of the advantages of evolutionary algorithms
listed in Goldberg (2005).
Coello (2007) presents the basic concepts and approaches of multi-objective opti-
mization evolutionary algorithms. The book further explores some hybrid methods
16
and introduces the test functions and there analysis. Various applications of multi-
objective evolutionary algorithms (MOEA) are also discussed in the book. Deb
(2001) is another comprehensive source of different MOEAs. The book divides the
evolutionary algorithms into non-elitist and elitist algorithms.
All researchers are agreed upon that the invention of first MOEA is devoted to
David Schaffer with his Vector Evaluation Genetic Algorithm (VEGA) in the mid-
1980s, aimed at solving optimization problems in machine learning.
Deb (2001) and Coello (2007) both name various MOEAs which shows the differ-
ence in the frame work and their operators as follows:
• Thermodynamical GA (TDGA)
Coello (2007) describes the concept of each EA along with an illustration of algo-
rithm and short notes on advantages and disadvantages. At the end he summarizes
all EAs in a table. While Deb (2001) devotes two complete chapter of the book
17
to fully define the concept and principle of each EA by step-by step description of
algorithm, hand calculation, discussion on advantages and short comings, calcu-
lating the computational complexity and simulating an identical test problem for
all algorithms.
Since there exist several MOEAs, a question of which algorithm has the best
performance is a common question among scientist and researchers. In order to
settle to an answer several test problems has been designed and various amount of
researches is carried out. In Deb’s book, a few significant studies on comparison
of EAs are discussed. (Deb, 2001)
Konak et al. (2006) demonstrates the advantages and disadvantages of most well-
known EAs in a table.
Zitzler et al. (2001) improves the original SPEA (Zitzler and Thiele, 1998), and
addresses some potential weaknesses of SPEA.
SPEA2 uses an initial population and an archive (external set). At the start,
random initial and archive population with fixed sizes are generated. The fitness
value of each individual in the initial population and archive is calculated per
iteration. Next, all non-dominated solutions of initial and external population are
18
copied to the external set of next iteration (new archive). With the environmental
selection procedure the size of the archive is set to a predefined limit. After wards,
mating pool is filled with the solutions resulted from performing binary tournament
selection on the new archive set. Finally, cross-over and mutation operators are
applied to the mating pool and the new initial population is generated. If any of
the stopping criteria is satisfied the non-dominated individuals in the new archive
forms the pareto-optimal set.
Kim et al. (2004) adds two new mechanisms to SPEA2 in order to improve the
searching ability of algorithm. The SPEA2 + algorithm, as it is named, uses a
more effective crossover (Neighborhood Crossover) and new archive mechanism to
diversify the solutions in both objective and variable spaces.
Kukkonen (2006) introduces a pruning method, which can be used to improve the
performance of SPEA2. The idea of pruning is to reduce the size of a set of non-
dominated solution to a pre-defined limit, while the maximum possible diversity
is encountered.
19
dominated individuals in Pt and P t , if size of P t+1 is less than N . (Environ-
mental Selection)
Fitness Assignment
X
R(i) = S(j).
j∈Pt +P t ,ji
The density estimation technique is adopted from the k-th nearest neighbor method
(Silverman 1986), where the density at any point is a (decreasing) function of the
distance to the k-th nearest data point. In SPEA2 the inverse of the distance to
the k-th nearest neighbor is considered as the density measurement. The density
D(i) corresponding to i is defined by:
1
D(i) = ,
σik +2
where,
p
k= N + N.
20
and σik is the distance of solution i to the k-th nearest neighbour. Finally, the
fitness of an individual i is calculated by adding D(i) to the raw fitness value
R(i):
Environmental Selection
The first step is to copy all non-dominated individuals, i.e., the ones with fitness
value lower than one, from archive and population to the external set of the next
generation:
If the size of non-dominated solutions is exactly the same as archive size (|P t+1 | =
N ) the environmental selection step is completed. Otherwise, there can be two
situations:
• The archive is too small (|P t+1 | < N ): The best N − |P t+1 | dominated
individuals in the previous external set and population are copied to the
new archive. This can be achieved by sorting the multi-set P t + P t+1 from
lowest to highest fitness values and copy the first N − |P t+1 | individuals i
with F (i) > 1 from the sorted list to P t+1 .
• The archive is too large (|P t+1 | > N ): In this case, an archive truncation
procedure is invoked which iteratively removes individuals from P t+1 until
(|P t+1 | = N ) . Here, at each iteration the solution i is chosen for removal
for which i ≤d j for all j ∈ P t+1 with:
21
In appendix B, hand calculation and step by step simulation of a simple
example minimization problem is fully described. This will help on better
understanding of algorithm and the working principle of each step.
|h(x)| − 6 0
Constraints divide the decision space into two separate parts: feasible and infea-
sible regions. A solution in the feasible region of search space satisfies all the
constraints and it is called a feasible solution, otherwise the solution is infeasible.
The most popular and common way of handling constraints is the penalty function
method. However sensitivity of penalty method to the penalty parameter is a
drawback in this method.
In Debs method a binary tournament selection operator is used for any two so-
lutions selected from the population. Therefore in presence of constraints three
scenarios will occur: 1) Both solutions are feasible; 2) One is feasible and the other
is infeasible; 3) Both solutions are infeasible. In the method for each scenario fol-
lowing rule is applied:
22
• Scenario 1) the solution with better objective function is selected (Crowded
comparison).
2. Solution i and j are both infeasible, but solution i has a smaller overall
constrained violation.
In the SPEA2 algorithm proposed for the thesis, binary tournament is applied to
the archive population (P t+1 ), which holds the non-dominated individuals to create
the mating pool. Afterwards the genetic operators are used to generate the child
from the mating pool which is the initial population of next generation (Pt+1 ). In
constrained problems the modified definition of domination is implemented and the
non-dominated solutions are selected according to constraint domination concept.
Appendix B.1 simulates the principle and procedures of constraint domination
concept on a simple problem.
23
2.4 Hybrid Multi-Objective Optimization Approach
In real world engineering problems there is no prior knowledge on the true global
pareto-front. Although Evolutionary algorithms have shown a good convergence
in benchmarks, hybrid methods have been proposed to ensure the convergence of
an algorithm to the true pareto-front. Several hybridization techniques (combining
an MOEA with other methods) are discussed in literatures.
Coello (2007) comprehensively deliberate the use of local search and co-evolutionary
techniques as a hybrid method in a complete chapter of his book. He specifies local
search decision space approaches such as depth-first search (hill-climbing), simu-
lated annealing and Tabu search for consideration in hybridization.
Deb (2001) also argues the use of local search techniques with an MOEA. Accord-
ing to Goldberg, the best way to achieve convergence to the exact pareto-front is
implementing the local search techniques on the solutions obtained from an EA.
However Deb proposes two other ways to use local search techniques; 1) during
EA generations, 2) at the end of an EA run.
First, the objectives are transformed to a single objective function by using the
weighted sum method. A number of random weights equal to size of population
are multiplied to each objective to scalarize the objective function. Then, every
scalarized function is optimized with G3 single objective GA. After finding the
optimal solution for each weighted function, the required initial population is
obtained. Finally, the multi-objective algorithm (SPEA2) is used to optimize the
function. Therefore, the hybridizing technique is applied before EA generations
to create the required initial archive population.
24
3 Implementation
All the algorithms in the thesis are coded with MATLAB. Several benchmarks are
encompassed and solved with the coded algorithms to ensure the accuracy and
efficiency of algorithm.
Sphere Function
n
X
fSphere (x) = x2i
i=1
Ellipsoidal Function
The behaviour of the algorithms for a poorly scaled objective function is discussed
using the following objective function:
n
X
fEllipsoid (x) = ax21 + x2i
i=1
Schwefel’s Function
n i
!2
X X
fSchwef el (x) = xi
i=1 j=1
25
Goldstein-Price Function
fGoldstein (x1 , x2 ) = (1 + (x1 + x2 + 1)2 )(19 − 14x1 + 3x21 − 14x2 + 6x1 x2 + 3x22 ))
×(30 + (2x1 − 3x2 )2 (18 − 32x1 + 12x21 + 48x2 − 36x1 x2 + 27x22 ))
Rosenbrock Function
This function is used to discuss the behaviour of the algorithms for functions
having complex non-separable structure, such as a curved, deep valley, given by
n
X
fRosenbrock (x) = (100(x21 − xi )2 + (1 − xi )2 ),
i=2
and has a global minimum of 0 at x∗i = 1. The typical search range is −5.12 ≤
xi ≤ 5.12, i = 1, ..., n.
Colville Function
∗
The search range is −10 ≤ xi ≤ 10 and the global minimum of fColville = 0 at
x∗i = 1, i = 1, ..., 4.
In addition, for Spherical, Ellipsoidal, Schwefel and Rosenbrock functions two cases
are considered for initial population:
26
• Normal Case: the distribution of initial population is surrounding the
optimal solution. The population is generated by uniform random numbers
in the region below:
−1 < xi < 1, i = 1, ..., n
• Offset Case: the distribution of initial population is faraway from the op-
timal solution.The population is generated by uniform random numbers in
the region below:
−10 < xi < −5, i = 1, ..., n
Test Function 1
The objective is to minimize the cost of the welded beam subject to the constraints
on shear stress (τ ), bending stress in the beam (σ), bucking load on the bar (Pc ),
end deflection of the beam (δ), and side constraints. The problem has four design
variables h(x1 ), l(x2 ), t(x3 ), b(x4 ) and five inequality constraints as follows:
27
Minimize f (x) = 1.1047x21 x2 + 0.04811x3 x4 (14.0 + x2 ),
Subject to g1 (x) = τM AX − τ (x) ≥ 0,
g2 (x) = σM AX − σ(x) ≥ 0,
g3 (x) = x4 − x1 ≥ 0,
g4 (x) = Pc (x) − P ≥ 0,
g5 (x) = δM AX − δ(x) ≥ 0,
0.1 ≤ x1 , x4 ≤ 2,
0.1 ≤ x2 , x3 ≤ 10
where
r q
0 0
τ (x) = ((τ (x))2 + (τ ” (x))2 + x2 τ (x)τ ” (x))/0.25 [x22 + (x1 + x3 )2 ],
6P L
σ(x) = ,
x23 x4
4P L3
δ(x) = ,
Ex23 x4
q
x23 x64 " r #
4.013E 36 x3 E
Pc (x) = 2
1− ,
L 2L 4G
where
0 P MR0
τ (x) = √ , τ ” (x) = ,
2x1 x2 J
r
h x2 i x22 x1 + x3 2
M =P L+ , R= +( ).
2 4 2
P = 6000 lb, L = 14 in, E = 30 × 106 psi, G = 12 × 106 psi,
τM AX = 13600 psi, σM AX = 30000 psi, δM AX = 0.25 in.
28
The optimized solution reported in literature (Deb, 1991) is x = (0.2489, 6.1730, 8.1789, 0.2533)
with f = 2.43 using binary GA.
140.45 ∗ x1
g3 (x) = − 1,
xx3 ∗ x22
x 1 ) + x2
g4 (x) = 1 − ,
1.5 − 1
0.05 ≤ x1 ≤ 2,
0.25 ≤ x2 ≤ 1.3,
2 ≤ x2 ≤ 15.
29
The best optimal solution obtained by using static penalty function is f ∗ =
0.012729 and the lowest optimal found by (Mezura-Montes et al., 2003) is f ∗ =
0.012688 using an approach based on Evolution Strategy.
Similar to single objective, two sets of test functions, one for unconstrained and
the other for constrained, are utilized to assess the performance of SPEA2 and the
proposed constrained handling method. The Algorithm is coded with MATLAB.
Exercise 14
2
f1 (x) = 1√− 2x + x ,
Minimize f2 (x) = x,
0 ≤ x ≤ 1.
30
Kursawe’s Test Function
Minimize F = (f1 (x), f2 (x)) ,
√
−0.2 x2i +x2i+1
f1 (x) = n−1
P
−10e
i=1 ,
KUR:
f2 (x) = ni=1 |xi |0.8 + 5 sin3i .
P
−5 ≤ xi ≤ 5, i = 1, 2, 3.
Deb (2001) illustrates the pareto-front of KUR function in figure 201. Three
distinct disconnected regions create the pareto-front of the problem. Also figure
202 of the same book shows the pareto-optimal solutions in decision space.
Zitzler et al. (2000) introduced six set of multi-objective problems (ZDT1 to ZDT6)
which are based on a unique structure with different level of difficulties. The
functions have two objective with the aim of minimization:
Minimize F = (f1 (x), f2 (x)) ,
ZDT: f1 (x),
f (x) = g(x)h(f (x), g(x)).
2 1
In the thesis five of the six test problems (ZDT1, ZDT2, ZDT3, ZDT4 and ZDT6)
are implemented with SPEA2 algorithm.
Minimize F = (f1 (x), f2 (x)) ,
f1 (x) = x1 ,
ZDT1: p
f 2 (x) = g(x) 1 − f 1 /g(x) ,
xi
g(x) = 1 + 9 ni=2
P
.
n−1
31
All the variables are limited between 0 and 1. Figure 213 of Deb (2001) shows
the search space and pareto-front in objective space. This is the easiest among all
ZDT’s and the only difficulty is the large number of variables.
Minimize F = (f1 (x), f2 (x)) ,
f1 (x) = x1 ,
ZDT2:
f2 (x) = g(x) 1 − (x1 /g(x))2 ,
9 Pn
g(x) = 1 + xi .
n − 1 i=2
The range for all the variables is [0, 1]. Pareto-front and the search region in
objective space is shown in figure 214 of Deb (2001). Non-convexity of pareto-
optimal set is the only difficulty of this problem.
Minimize F = (f1 (x), f2 (x)) ,
f1 (x) = x1 ,
ZDT3: p
f 2 (x) = g(x) 1 − f 1 /g(x) − (f1 /g(x)) sin (10πf1 ) ,
x i
g(x) = 1 + 9 ni=2
P
.
n−1
All the variables are limited within [0, 1]. Finding all the discontinuous pareto-
optimal regions with a good diversity of non-dominated solutions may be difficult
for an MOEA. Deb (2001) show the search space and pareto-front in figure 215.
32
ZDT4 Test Function
Minimize F = (f1 (x), f2 (x)) ,
f1 (x) = x1 ,
ZDT4: p
f 2 (x) = g(x) 1 − x 1 /g(x) ,
g(x) = 1 + 10 (n − 1) + ni=2 (x2i − 10 cos (4πxi )) .
P
All the variables except x1 , which lies in the range [0, 1], are limited within −5
and 5. Large number of multiple local pareto-fronts, shown in figure 216 of Deb
(2001), will create a difficult convergence to global pareto-front for an MOEA.
Minimize F = (f1 (x), f2 (x)) ,
f1 (x) = 1 − exp(−4x1 ) sin6 (6πx1 ),
ZDT6: f 2 (x) = g(x) 1 − (f 1 /g(x))2
,
1/4
xi
Pn
g(x) = 1 + 9 .
i=2
n−1
All the variables lie in the range [0, 1]. Non-convexity of pareto-front, coupled with
adverse density solutions across the front, may rise some difficulty in convergence.
Figure 218 in Deb (2001) shows the pareto-optimal region for this problem.
The presence of constraints may cause hurdles for an MOEA to converge to the true
and global pareto-front, also maintaining diversity in the non-dominated solutions
may be another problem. A number of common test problems used in literatures,
are presented in this section and implemented in the SPEA2 code.
33
Binh and Korn Test Function
Minimize f1 (x) = 4x21 + 4x22 ,
f2 (x) = (x1 − 5)2 + (x2 − 5)2 ,
Minimize
C1 (x) = (x1 − 5)2 + x22 ≤ 25,
subject to
BNH:
C2 (x) = (x1 − 8)2 + (x2 + 3)2 ≥ 7.7,
0 ≤ x1 ≤ 5,
0 ≤ x2 ≤ 3.
Deb (2001) illustrates the decision variable and objective space of the problem in
figures 219 and 220. In BNH problem, constraints will not add any difficulty to
the unconstrained problem.
Osyczka and Kundu (1995) used the following six variable test function:
Minimize f1 (x) = −[25(x1 − 2)2 + (x2 − 2)2 + (x3 − 2)2 + (x4 − 2)2
+(x5 − 2)2 ],
x21 + x22 + x23 + x24 + x25 + x26 ,
Minimize f2 (x) =
subject to C1 (x) = x1 + x2 − 2 ≥ 0,
C2 (x) = 6 − x1 − x2 ≥ 0,
OSY: C3 (x) = 2 + x1 − x2 ≥ 0,
C4 (x) = 2 − x1 + 3x2 ≥ 0,
C5 (x) = 4 − (x3 − 3)2 − x4 ≥ 0,
C6 (x) = (x5 − 3)2 + x6 − 4 ≥ 0,
0 ≤ x1 , x2 , x6 ≤ 10,
1 ≤ x3 , x5 ≤ 5, 0 ≤ x4 ≤ 6.
The pareto-front as shown in figure 221 of Deb (2001), is a line connecting some
parts of five different region. Since the algorithm should maintain the solutions
within intersections of constraint boundaries, this is a difficult problem to solve.
34
Srinivas and Deb Test Function
Minimize f1 (x) = 2 + (x1 − 2)2 + (x2 − 1)2 ,
Minimize f2 (x) = 9x1 − (x2 − 1)2 ,
SRN: subject to C1 (x) = x21 + x22 ≤ 225,
C2 (x) = x1 − 3x2 + 10 ≤ 0,
−20 ≤ x1 , x2 ≤ 20.
Since the constraints eliminate some parts of the original pareto-front, difficulties
may arise in solving the problem. Figures 222 and 223 in Deb (2001) shows the
corresponding pareto-front of feasible decision variable and objective space.
Tanaka and Watanabe (1995) proposed a two variable test function as follows:
Minimize f1 (x) = x1 ,
Minimize f2 (x) = x2 ,
x1
TNK: subject to x21 x22
C1 (x) = + − 1 − 0.1 cos 16 arctan ≥ 0,
x2
2 2
C2 (x) = (x1 − 0.5) + (x2 − 0.5) ≤ 0.5,
0 ≤ x1 , x2 ≤ π.
35
3.3 Hybrid Approach
In order to test the performance of the proposed hybrid method, the archive pop-
ulation generated from the hybrid technique (weighted sum single objective G3
algorithm) is compared with the random archive population for different test prob-
lems. It is obvious that the test functions with non-convex pareto-fronts are not
suitable for the technique, since the weighted sum method is used to transform
the objectives into one objective. Therefore, the random and hybrid archive pop-
ulation are plotted in objective space for ZDT1 and ZDT3 test functions. Also
despite the non-convex property of ZDT6 problem, comparison of archive popu-
lation has also been applied to this problem to study the performance of hybrid
method on non-convex benchmarks.
36
4 Test Results
To examine the behaviour of algorithm and code, evaluation in two and multi-
dimensional search space is carried out for some of the test functions as blow:
• Spherical: n = 2, 4
• Ellipsoidal: n = 2, 4
• Schwefel: n = 2, 10, 15
• Rosenbrock: n = 2, 5
37
The experiment for each function runs until the best objective function of the pop-
ulation reaches a minimum difference of 10−5 from the optimal solution. Number
of Generation (evaluation) and best fitness are shown in table 1.
The result shows acceptable behaviour of algorithm for two dimensional search
space with ση = 0.1 and σζ = 0.1 , but when the number of variable increases
or functions are more complex such as Rosenbrock, the algorithm converges in
local optima or the global optima is obtained with high number of generations.
Therefore by increasing the variance of zero-mean normally distributed variables
in PCX operator better results are obtained as it can be seen in table of results
the Schwefel’s function with 15 variable has reached the required variance from
global optimal. Convergence of the best individual obtained from some of the test
functions during generations can be seen in the figures 5 and 6.
38
4.1.2 Constrained Functions
The Penalty method used for constrained test functions shows a good behaviour.
All three functions reached the optimal solution reported in literature, in ad-
dition the optimal solution found by the algorithm in this thesis with related
constraint handling method is better than some other approaches used in liter-
ature. In the first test function the optimal solution of 13.590842 is found at
x∗ = (2.246818, 2.381735) which is better than the optimal found at literature
(Deb, 2000) with the value f ∗ = 13.59085. Furthermore, figures 7, 8 and 9 illus-
trate the convergence of results for the three constrained test functions.
Tables 2 and 3 compare the results of different methods for welded beam de-
sign problem and the minimization of the weight of a tension/compression spring,
which shows the out-performance of the approach used here to some methods.
Figure 8: Convergence of objective function for welded beam problem with ob-
tained optimal of f ∗ = 1.834756
39
Table 2: Comparison of the results of different methods for welded beam design
problem.
Method f ∗ (x)
This Thesis 1.83475678
Coello (self-adaptive penalty approach) 1.74830941
Arora (constraint correction at constant cost) 2.43311600
He and Wang (CPSO) 1.728024
Ragsdell and Phillips (Geometric programming) 2.385937
Deb (GA) 2.433116
Coello and Montes (feasibility-based tournament selection) 1.728226
Ebehart (modified PSO) 1.72485512
Method f ∗ (x)
This Thesis 0.012710175
Coello (self-adaptive penalty approach) 0.01270478
Arora (constraint correction at constant cost) 0.12730274
He and Wang (CPSO) 0.0126747
Belegundu (numerical optimization technique) 0.0128334
Coello and Montes (feasibility-based tournament selection) 0.0126810
Ebehart (modified PSO) 0.01266614
40
4.2 Multi-Objective
SPEA2 algorithm coded with MATLAB is used to solve multi-objective test func-
tions. Simulated Binary Crossover (SBX) and Polynomial Mutation (Deb, 2001)
are implemented in the step 6 (Variation) of SPEA2 algorithm as recombination
and mutation operators.
In exercise 14 test function, a weight vector with 20 weight factors within the range
[0, 1] with step length of 0.05 is used to create 20 single objective functions and
each function is optimized separately to obtain the pareto-front shown in figure
10 . In multi-objective method, the initial and archive set with population of 30
individuals after 100 generations result in the pareto-front (figure 11).
41
Since the function is non-convex the pareto-front obtained from single objective
method does not cover the non-convex parts of pareto-optimal set. In the other
hand, the pareto-front of multi-objective method clearly illustrates all parts of
pareto-optimal region. Thus, an important drawback of single-objective weighted
sum method for solving multi-objective optimization problems is the weakness in
non-convex problems.
Table 4 shows the defined parameters of SPEA2 algorithm, such as size of initial
and archive population and number of generations, for the other test functions
from Kursawe to ZDT6.
Figures 12, 13, 14, 15, 16 and 17 shows the non-dominated solutions obtained
from SPEA2 algorithm for problems Kursawe, ZDT1, ZDT2, ZDT3, ZDT4 and
ZDT6. Comparing the figures to the true pareto-fronts illustrated in literature
(Deb, 2001), confirms the excellent performance of SPEA2 algorithm in finding
the pareto-front of problems with upto 30 variables.
42
Figure 13: Pareto-front of ZDT1 test function
Throughout this thesis, all the multi-objective test functions contain two objec-
tives, thus the performance of an algorithm can be measured and illustrated with
representing the non-dominated solutions in a two-dimensional objective space
43
plot. However, illustrating the non-dominated solutions in a multi-objective prob-
lem with more than two objectives can be a difficult task. Even the 3D plot for
three objective problems, which each axes represents one objective, is confusing
and unhelpful.
There are number of methods for presenting problems with more than two ob-
jectives in literatures. Scatter-plot matrix is one way, which Meisel (1973) and
Cleveland (1994) suggest to plot all (M
2 ) pairs of plots among the M objective
functions. Therefore, the non-dominated solutions of a problem with three ob-
jectives will be illustrated with 6 plots in a 5 × 5 matrix. Each diagonal plot is
used to mark the axis for the matching off diagonal plots. In this method the
non-dominated solutions in each pair of objective spaces are shown twice with the
difference in the axis marked for each objective.
44
The scatter plot matrix can also be used for comparison of two different algorithms
on an identical problem. The upper diagonal plots shows the non-dominated
solutions of one algorithm and lower diagonal plot is utilized to illustrate the
corresponding solutions of other algorithm.
45
46
Figure 18: Scatter-plot matrix of Kursawe test function
47
Figure 19: Scatter-plot matrix of ZDT1 test function
48
Figure 20: Scatter-plot matrix of ZDT2 test function
49
Figure 21: Scatter-plot matrix of ZDT3 test function
50
Figure 22: Scatter-plot matrix of ZDT4 test function
51
Figure 23: Scatter-plot matrix of ZDT6 test function
4.2.2 Constrained Functions
The constrained test functions are optimized by SPEA2 algorithm with the pre-
defined parameters shown in table 5. The non-dominated solutions obtained for
BNH, OSY, SRN and TNK problems are illustrated respectively in figures 24, 25,
26 and 27.
52
Comparing the figures with the true pareto-fronts reported in literatures, proves
the good performance of algorithm in converging to optimal results with a good
diversity of solutions.
53
4.3 Hybrid Approach
Figures 28 and 29 illustrate the comparison of the two archive population for ZDT1
and ZDT3 test functions respectively.
Assessing the plots with the existing true pareto-optimal fronts in the literatures,
shows that the hybrid approach generates a population near the actual pareto-
front.
However, figure 30 which plots the two population of ZDT6 problem confirms the
fact that convexity of objective function has an important influence in diversity
and closeness of population to pareto-front.
Furthermore, the obtained hybrid populations are the outcome of a single objective
GA with relatively high number of generations. Consequently, the proposed hybrid
approach do not show any improvement in overall computation time of test func-
tions. However the number of generations to reach near the actual pareto-front,
accordingly the computation time in the multi-objective part of the algorithm
decreases.
Figure 28: Hybrid (left) and random (right) initial archive population for ZDT1.
Figure 29: Hybrid (left) and random (right) initial archive population for ZDT3.
54
More precise and reliable judgement can be made only after conduction of an
extensive research on convergence of optimization problems and introducing a
proper metric to compare the two approaches in a more scientific way.
Also, parameter setting in the hybrid method will have an important effect on
computation time. There exist a large number of parameters including size of
population in each algorithm, number of generations for single objective algorithm
and size of different sets used in the algorithm, which will have a great impact
in computation time. Nevertheless, creating a predefined archive population may
enhance convergence to the true pareto-front in an EA.
Figure 30: Hybrid (left) and random (right) initial archive population for ZDT6.
55
5 Conclusion
After implementing the proposed algorithm for single objective optimization test
functions, it was concluded that the approach showed a good performance in con-
verging to the true optimal solution. However parameter setting in problems with
higher number of variables is crucial. The penalty method used for constrained
handling managed to find the optimal solution for all three test functions. Also,
better behaviour was observed in comparison to some of the other techniques.
The scatter-plot matrix method for illustrating the non-dominated solutions, could
be very supportive in studying real world engineering problems, where understand-
ing the relations between variables and objectives are crucial.
The suggested hybrid approach did not show any advantages in overall computa-
tion time, and in some problems it can be considered as a weakness regarding this
issue. There is an essential need of comprehensive studies related to convergence
of optimization problems, comparison metrics and different ways of combining
single and multi-objective methods in order to conclude in a more precise and
scientific manner. It is believed that the hybrid method may improve the ability
of algorithm in finding the global optimal solutions.
56
6 Bibliography
Beasley, D. and Bull, D. R. (1993). An Overview of Genetic Algorithms : Part 1
, Fundamentals 1 Introduction 2 Basic Principles. Building, pages 1–16.
Corne, D., Jerram, N., and Knowles, J. (2001). PESA-II: Region-based selection
in evolutionary multiobjective optimization. Genetic and Evolutionary.
Deb, K. (1991). Optimal design of a welded beam structure via genetic algorithm.
AIAA.
57
Deb, K., Joshi, D., and Anand, A. (2001). Real-coded evolutionary algorithms with
parent-centric recombination. Technical report, KanGAL Report No. 2001003,
Kanpur Genetic Algorithms Laboratory (KanGAL), Indian Institute of Tech-
nology, Kanpur.
Deb, K., Pratap, A., Agarwal, S., and Meyarivan, T. (2002). A fast and elitist
multiobjective genetic algorithm: Nsga-ii. IEEE Transactions on Evolutionary
Computation, 6(2):182–197.
Gen, M. and Cheng, R. (1997). Genetic algorithms and engineering design. New
York: Wiley.
Kim, M., Hiroyasu, T., Miki, M., and Watanabe, S. (2004). Spea2+: Improving
the performance of the strength pareto evolutionary algorithm 2. 3242:742–751.
Knowles, J. (1999). The pareto archived evolution strategy: A new baseline algo-
rithm for pareto multiobjective optimisation. Evolutionary Computation, 1999.
CEC.
58
Knowles, J. (2000). Approximating the nondominated front using the Pareto
archived evolution strategy. Evolutionary computation, pages 1–35.
Konak, a., Coit, D., and Smith, a. (2006). Multi-objective optimization us-
ing genetic algorithms: A tutorial. Reliability Engineering & System Safety,
91(9):992–1007.
Kung, H. and Luccio, F. (1975). On finding the maxima of a set of vectors. Journal
of the ACM (JACM), (4):469–476.
Man, K. F., Tang, K. S., and Kwong, S. (1996). Genetic Algorithms : Concepts
and Applications. October, 43(5).
59
Stromberg, N. (2011). Nonlinear FEA abd Design Optimization for Mechanical
Engineers. Jonkoping University, 2nd edition.
Zitzler, E., Deb, K., and Thiele, L. (2000). Comparison of multiobjective evolu-
tionary algorithms: Empirical results. Evolutionary computation, 8(2):173–195.
Zitzler, E., Laumanns, M., and Thiele, L. (2001). SPEA2: Improving the strength
Pareto evolutionary algorithm. Computer Engineering, pages 1–21.
60
Appendices
A simple real world single objective with constraint optimization problem is pre-
sented and solved by the proposed G3 algorithm and related constraint handling
method for one generation run.
A car spare part manufacturing company manufactures disk brakes and brake
pads. A disk brake takes 8 hours to manufacture and 2 hours to finish and pack.
A brake pad takes 2 hours to manufacture and 1 hour to finish and pack. The
maximum number of labour-hours per day is 400 for the manufacturing process
and 120 for the finishing and packing process. If the profit on a disk brake is
90Euro and the profit on a brake pad is 25Euro, how many disk brakes and brake
pads should be made each day to maximize the profit (assuming that all of the
disk brakes and brake pads can be sold)?
Where x1 is the number of disk brakes and x2 is the number of brake pads. The ob-
jective can be transformed to a minimization problem by multiplying the objective
function by −1. The constraints are labour hours for each product:
Minimize −90x1 − 25x2 ,
subject to 8x + 2x − 400 ≤ 0,
1 2
Spare part company:
2x1 + x2 − 120 ≤ 0,
x1 , x2 ≥ 0.
The G3 algorithm has four steps (plans): 1) Selection plan, 2) Generation Plan,
61
3) Replacement plan, 4) Update plan. A population of random solutions named
set B, with 10 individuals is created. Table 6 shows set B and the initial random
solutions. Other pre-defined parameters are:
µ = 3, λ = 2, r = 2.
Solution x1 x2 Fitness
1 64.56 20.46 109
2 77.39 28.45 109
3 26.70 63.96 −4002.47
4 19.05 11.10 −1992.25
5 97.83 8.99 109
6 80.16 74.53 109
7 33.28 34.16 −3849.32
8 33.97 62.01 5 × 108
9 21.43 48.04 −3129.69
10 68.42 75.20 109
Step 1: The best solution in set B and µ − 1 other random solutions create set
P . First, the best solution, with regards to its fitness, is chosen. Thus, the
fitness (value of objective function) of each individual in set B has to be
calculated. Since a constraint is involved, the fitness is assigned according
to the proposed constraint handling method:
F (3) = −4002.47
62
where K is a large enough pre-defined penalty factor (K = 109 ), s
is the number of constraints (s = 2) and m is the number of non-
violated constraints. Solution 8, is infeasible and only satisfies the first
constraint, therefore:
m = 1, F (8) = 5 × 108 .
~g = [67.31, 33.80]
Step PCX2: For each offspring one parent is randomly selected from
set P . For instance, the first randomly selected parent is solution
2. The direction vector is obtained:
d~ = ~g − ~x = [10.08, −5.35]
~e = [0.47, 0.88]
63
wζ2 = 0.5 and wη2 = 0.51 are:
wη = 0.17, wζ = 0.17.
The two randomly selected solutions in step 3, solutions 2 and 7, are re-
placed with the first r (r = 2) solutions from R∪Csorted , in set B. There-
fore set B is modified to Bnew = {1, of f spring2, 3, 4, 5, 6, 2, 8, 9, 10} and
the first generation is completed. Next generation starts with the new
set Bnew , from step 1.
1
The value of variance is selected according to the desired distance of offspring from parent.
In other words, higher values of variances increases the distance of offspring from parent, whereas
a small variance creates an offspring close to the parent.
2
The fitness of the two newly created offspring is calculated according to the same procedure
described in steps C1 to C3 considering the feasibility or infeasibility of offspring.
64
B Hand Calculation of SPEA2 Algorithm
Minimize f1 (x) = x1 ,
Minimize f (x) = x2 + 1 ,
2
Min-Ex: x1
subject to 0.1 ≤ x1 ≤ 1,
0 ≤ x2 ≤ 5.
This problem unlike the simple look, has two conflicting objective which
create a convex pareto-front as shown in figure 31. The search space is also
illustrated in the figure.
The SPEA2 algorithm has two initial sets; one the initial population and the
other the external set which holds the non-dominated solutions. In order to
show the working principle of algorithm, six random solutions in the search
space is chosen for the first set and three random solutions for the archive set,
as it is done in the code. These solutions and their corresponding objective
function values are listed in table 7.
Step 1: The two initial random populations are created. The solution in
65
Table 7: Current and external initial random population of SPEA2 with their
objective function values
66
puted: v
u |M | !2
(i) (j)
uX fk − fk
dij = t
k=1
fkmax − fkmin
1
D(1) = = 0.4974
σ13 +2
Step F4: Fitness value of each individual is calculated. Fitness of
solution 1 is:
Density and fitness values of all solutions are listed in the two last
columns of table 8.
67
Table 8: Fitness assignment procedure of current and external set of SPEA2
Initial population Pt
Solution S(i) Dominated Solutions R(i) F (i)
1 2 2, 4 0 0.4974
2 1 4 2 2.4928
3 1 4 0 0.4974
4 0 6 6.497
5 2 6, 8 0 0.4972
6 0 3 3.4912
External population Pt
Solution S(i) Dominated Solutions R(i) F (i)
a 1 4 0 0.4992
b 1 6 2 2.4948
c 1 4 0 0.4980
Step E2: After sorting the distances in increasing order, the k th near-
est solution (here we set k = 2) to solution 1 is solution a with
d1a = 0.0022. The same procedure is done for all other solutions.
With comparing the 2nd (k th ) nearest solution to all solutions in
P1 , we can conclude that solutions a and c should be eliminated.
They have the lowest distance to their 2nd nearest solution; there-
fore they are removed from the archive set to improve the diversity
of non-dominated solutions.
Step 4: Since the stopping criteria which is the number of generations is
not met, we will continue to step 5.
Step 5: In this step the three individuals in P1 will participate in a binary
tournament selection with replacement. Each solution will participate
in two tournaments and the better solution in matter of lower fitness will
win the tournament and be placed in the mating pool. Since solution
5 has the best fitness among the three, it will win both tournaments,
thereby creating two copies of it in the mating pool. Therefore the
mating pool is filled with solutions {5, 5, 3}.
68
Step 6: The variation step creates child population from the parent popu-
lation of mating pool. Here we used cross-over probability equal to 0.9
and mutation probability of 0.1.
Step V1: Two parents are randomly selected; we assume that solution
5 is the first parent and 3 is the second one.
Step V2: Two children are created from the parents by using SBX
cross-over and pre-defined value of ηc = 2 as follows:
Step SBX1: A random number ui ∈ [0, 1] is chosen. For example
u1 = 0.7577.
Step SBX2: βqi is calculated by:
1
(2ui ) ηc + 1 , ifui ≤ 0.5;
βqi =
1
1
ηc + 1
, otherwise.
2(1 − ui )
βq1 = 1.2732
(1,t) (2,t)
here, x1 = 0.66 and x1 = 0.22 therefore the children are:
(1,t+1) (2,t+1)
x1 = 0.7201, x1 = 0.1599.
The same procedure with a random u2 = 0.72 is done for the second
variable and the resulted children are:
(1,t+1) (2,t+1)
x2 = 1.5157, x2 = 0.4543.
69
Step V3: By keeping in mind the mutation probability, mutation of
a randomly selected parent is achieved with polynomial mutation
operator and predefined parameter ηm = 2 :
Step PBX1: A random number ri ∈ [0, 1] is chosen. For example
ri = 0.7.
Step PBX2: Parameter δi is calculated:
(2ri ) ηm1+1 − 1, if ri < 0.5,
δi = 1
1 − [2(1 − ri )] ηm +1 , if ri ≥ 0.5.
1
Since r1 = 0.7 > 0.5 therefore; δ1 = 1 − [2(1 − 0.7)] 2+1 = 0.1566
Step PBX3: The new mutated child is computed with:
(1,t+1) (1,t+1)
yi = xi + δi .
70
B.1 Constraint Handling Method of SPEA2 Algorithm
The same simple minimizing optimization problem used for hand calculation of
SPEA2 algorithm is used to simulate the working principle of suggested constraint
handling method. However, two constraints are added to the problem:
Minimize f1 (x) = x1 ,
x2 + 1
Minimize f2 (x) = ,
x1
Min-Ex: subject to x2 + 9x1 ≥ 6,
−x2 + 9x1 ≥ 1,
0.1 ≤ x1 ≤ 1,
0 ≤ x2 ≤ 5.
Constraints divide the decision and objective search space into two regions. A part
of the unconstrained pareto-optimal front is not feasible and a new pareto-front
will emerge. The new constrained pareto-front is convex. The same two initial
sets, six solutions for the first set and three solutions for the external set (table 7),
are chosen. Figure 32 illustrates the constrained pareto-front and the solutions in
objective space.
Figure 32: Constrained Min-Ex pareto-front, feasible region and initial solutions
The only difference of constrained SPEA2 algorithm with the normal algorithm
is the definition of domination concept. Therefore, a step by step simulation of
constraint domination concept is described by using the initial chosen population.
71
Step CD1: Feasibility or infeasibility of every solution in current and external
set is examined.
Step CD2: The three scenarios explained in constraint handling method are in-
vestigated for each pair of solutions:
72