Anda di halaman 1dari 51

Paralllisation massive d'algorithmes volutionnaires sur cartes GPU avec EASEA

Frdric Krger, Pierre Collet Universit de Strasbourg


1

What is EASEA ?
!! EASEA = EAsy Specification of Evolutionary Algorithms !! Specification language based on C++ !! Goals: !! Allow anyone to design their evolutionary algorithm without having to actually program it. !! Allow to share EASEA source codes for reproducibility of results.

Types of algorithms supported by EASEA


!! EASEA can generate: !! Evolutionary Algorithms CAN BE PARALLELIZED ON GPU ! !! GP CAN BE PARALLELIZED ON GPU ! !! Memetic Algorithms CAN BE PARALLELIZED ON GPU ! !! EASEA can also implement complex island model structures based on the EAs cited above. !! French EASEA-CLOUD ANR project to port EASEA on the Grid and on the Cloud ($500K over 2 years).

Standard EASEA generated EA


Population initialisation evaluation Parents Stop ? Best Individual

Replacement

Generations

Children creation

Selection of n parents

Strong elitism

Weak elitism

Variation operators (Xover + mutation)!

Parents +children

evaluation

Processing of an EASEA project


EASEA EASEA compiler .ez Templates Linked to C++

EASEA Library

How does the EASEA compiler work ?


!! Takes an EASEA (or .ez) file as input. !! Selects the correct template (depending on the required evolutionary engine) !! Uses the contents of the .ez file to fill the correct template with the problem-dependent user code. !! Generates a C++ or CUDA source code. !! Source code only contains problem specific functions. !! Links the generated code to the EASEA library.

What are the EASEA templates ?


!! Algorithms with holes to be filled with problemdependent user code !! Different templates because EASEA can implement different types of Evolutionary Algorithms !! Templates + user code are split into several source files when compiled by EASEA !! Because the template is man-made, the result is readable (indented and commented) code ! !! Can by changed by daring users.

Existing templates
!! Template for standard EAs (Genetic Algorithms and Evolution Strategies) !! CPU version (C++) !! GPU version (C++ and CUDA) !! Template for GP !! CPU and GPU versions !! Template for Memetic Algorithms !! CPU and GPU versions !! Template for CMA-ES !! CPU version only
8

EASEA library
!! The EASEA library contains: !! Evolutionary loops !! Population management functions !! Selection operators !! Statistical functions !! Utility functions (display/plot functions) !! Island communication functions !! User readable !! Can be used without the EASEA compiler for custom projects
9

Structure of an EZ file
!! Structured in different layers or fields: !! Individual specific fields !! User definition fields !! Interaction fields !! Parameters fields !! Uses specific keywords and anchors (Genome, )

10

Main evolutionary sections to be provided


!! !! !! !! !! Genome definition Initialization function Evaluation function Crossover function Mutation function

11

Example of an EZ file

12

Genome structure

\User classes :! !GenomeClass { ! ! !float x[SIZE];! !}! \end!

13

Genome initialiser

\GenomeClass::initialiser:! !for(int i=0; i<SIZE; i++){! ! !Genome.x[i] = random(-1.0, 1.0);! !}! \end!

14

Genome evaluator (example for Weierstrass)


\GenomeClass::evaluator:! !float val[SIZE];! !float res = 0., b = 2., h = 0.25;! !for(int i=0; i<SIZE; i++){! ! !val[i] = 0.;! ! !for (int k=0; k<ITER; k++)! ! ! !val[i] += pow(b,-(float)k*h) * ! ! ! ! !sin(pow(b,(float)k)*Genome.x[i]);! ! !res += Abs(val[i])! !}! !return res;! \end!
15

Genome crossover (barycentric)

\GenomeClass::crossover:! !for(int i=0; i<SIZE; i++){! ! !float alpha = random(0.,1.);! ! !child.x[i] = alpha * parent1.x[i] ! ! ! ! ! ! !+ (1.-alpha) * parent2.x[i];! !}! \end!

16

Genome mutator

\GenomeClass::mutator:! !int nbMut = 0;! !for(int i=0; i<SIZE; i++){! ! !if (tosscoin(fMUT_PER_GEN)){! ! ! !NbMut++;! ! ! !Genome.x[i] += random(-0.1, 0.1);! !}! !return NbMut;! \end!
17

User fields
User declarations
\User declarations:! !! \end!

User Classes
\User classes:! !Element {! ! !int Value;! ! !Element *pNext; }! !GenomeClass {! ! !Element *pList;! ! !int Size;}! \end!
18

User functions
\User functions: ! !! \end!

Interaction fields
Population initialisation evaluation Parents Stop ? Best Individual

Replacement

Generations

Children creation

Selection of n parents

Strong elitism

Weak elitism

\Before everything else function:! ! \end!


Parents +children

Variation operators (Xover + mutation)!

evaluation

19

Interaction fields
Population initialisation evaluation Parents Stop ? Best Individual

Replacement

Generations

Children creation

Selection of n parents

Strong elitism

Weak elitism

\After everything else function: ! operators (Xover + ! mutation)! \end!


Parents +children evaluation

Variation

20

Interaction functions
Population initialisation evaluation Parents Stop ? Best Individual

Replacement

Generations

Children creation

Selection of n parents

Strong elitism

Weak elitism

\At the beginning of each generation function: ! operators (Xover + ! mutation)! \end!
Parents +children evaluation

Variation

21

Interaction functions
Population initialisation evaluation Parents Stop ? Best Individual

Replacement

Generations

Children creation

Strong elitism

Weak elitism

\At every generation before reduce function:! Selection of ! n parents \end!


Variation operators (Xover + mutation)!

Parents +children

evaluation

22

Interaction functions
Population initialisation evaluation Parents Stop ? Best Individual

Replacement

Generations

Children creation

Selection of n parents

Strong elitism

Weak elitism

\At the end of each generation function: ! ! Parents evaluation \end! +children

Variation operators (Xover + mutation)!

23

Parameters section
!! Parameters section defines the complete engine of the EA. !! A parameters file (.prm) is generated along with the source code files. !! All parameters can be redefined !! In the PRM file. !! As command line at the execution.

24

Main parameters
!! Default stopping criteria: !! Number of generations !! Time limit !! Population management: !! Population size !! Number of offspring !! Surviving offspring !! Number of elite individuals !! Etc.
25

Available selection operators


!! Syntax:
!!

Selection operator: OPERATOR SELECTION_PRESSURE!

!! Operators: !! Tournament (with selection pressure) !! Deterministic !! Roulette (only when maximizing) !! Random

26

Other parameters
!! Printing/Plotting statistic: !! Print stats : true! !! Plot stats : true !! Save population/Start from population: !! Save population: true !! Start from file: true !! User defined parameters

27

Predefined EASEA functions and variables


!! The EASEA language has a number of predefined functions and variables: !! Random number generators. !! Pointer to the Evolutionary Algorithm engine. !! Pointer the parent population. !! Pointer to the current best individual. !! Engine related variables (NB_GEN, currentGen, etc)

28

Parallelizing on GPU with EASEA


!! The same EASEA code can be compiled for sequential execution on CPU or parallelization on GPU !! The current version of EASEA only parallelizes the evaluation function on GPU !! The rest of the EA is kept on the CPU. !! To parallelize: !! Add -cuda parameter when compiling with EASEA !! easea project.ez cuda !

29

How it works
!! Population is sent from the CPU memory to the GPU global memory !! Individuals are dispatched between blocks and evaluated in parallel !! The fitness of an individual is sent back from the GPU global memory to the CPU memory, and reassigned to the individual in the CPU memory space

30

Standard EASEA sequential EA on CPU


Population initialisation evaluation Parents Stop ? Best Individual

Replacement

Generations

Children creation

Selection of n parents

Strong elitism

Weak elitism

Variation operators (Xover + mutation)!

Parents +children

evaluation

31

Parallelization of an EASEA EA on one GPU


Population initialisation evaluation evaluation evaluation Best Individual Parents Stop ?

Replacement

Generations

Children creation

Selection of n parents

Strong elitism

Weak elitism

Variation operators (Xover + mutation)!

Parents +children

evaluation evaluation evaluation

32

Parallelization over three GPU cards


Population initialisation evaluation evaluation evaluation evaluation evaluation evaluation Parents Stop ? Best Individual

evaluation evaluation evaluation

Replacement

Generations

Children creation

Selection of n parents

Strong elitism

Weak elitism

Variation operators (Xover + mutation)!

Parents +children

evaluation evaluation evaluation

evaluation evaluation evaluation

evaluation evaluation evaluation

33

Weierstrass function

34

Speedup on Weierstrass with 1x GTX580


x284.5

35

GP with EASEA
!! Each individual is represented as a tree !! Evaluation is done over a training set !! The user specifies how to use the difference between the calculated value and the training set value, to create a fitness value. !! Minimum and maximum depths are specified in the parameters section as well as crossover max depth,

36

Parallelization of GP over GPUs


!! GPUs use SIMD multiprocessors (32 cores must execute the same instruction at the same time). !! This is not a problem for EAs (identical evaluation function for different individuals). !! Very problematic for GP where individuals implement the evaluation code. !! The EASEA solution is then to parallelize the evaluation of one individual over the training set (ideally more than 32 values, to benefit from temporal pipelining parallelization). !! GP parallelization is spatio-temporal
37

GP Speedup curves: x100 x250

38

Memetic algorithms with EASEA


!! Memetic algorithms combine a global search algorithm with a local search routine. !! Newly created individuals are locally improved before taking part in the replacement process. !! Implementation in the EASEA language: !! Specific Genome section: Genome optimizer !! Memetic specific parameters:
! Number of local search iterations ! Baldwinian/Lamarckian evolution
!!

EASEA takes care of generating the local optimization loop.


39

Parallelizing a Memetic Algorithm with EASEA


!! Only the local optimization loop is parallelized on the GPU. !! The rest of the Memetic Algorithm is kept on the CPU. !! GPU returns: !! Complete locally improved population: Lamarckian evolution. !! Or only the improved fitness: Baldwinian evolution. !! To parallelize a project with EASEA: !! easea memetic.ez memetic cuda

40

EASEA parallelized Memetic Algorithm


Population initialisation local optimization local optimization local optimization evaluation evaluation evaluation Best Individual Stop ?

NX

Parents

Replacement

Generations

Children creation

Selection of n parents

Strong elitism

Weak elitism

Variation operators (Xover + mutation)! local optimization local optimization local optimization

Parents +children

evaluation evaluation evaluation

NX
41

GPU acceleration on zeolite pb


~120 with GTX275

42

EASEA island model


!! Combining the power of several EAs. !! Asynchronous machines can work together. !! EASEA island model allows: !! Network of heterogeneous machines !! Network of heterogeneous algorithms !! EASEA island model only requires a text file containing the IPs and ports of the remote machines
! IP:Port -> 192.168.0.2:2929

!! Relies on the UDP communication protocol.

43

EASEA island model

44

Benchmark function: 1000 dim Rastrigin

45

Multi-island acceleration on Rastrigin

Same pop size !

20 machines

10 machines

5 machines

1 machine

46

Speedup on Weierstrass function


!"#$%#"&'()*+''(%+) ,-./')&0)1'"$2)&2')*"/')3&4'**)5.&2)(.6'1'4&)$#%*&'1) *.7'*8)
90 80 70 60 Speedup factor 50 5 Machines vs. 1 Machine 40 30 20 10 0 """!# ")"!# ("!# '"!# &"!# %"!# $"!# !"!# Weierstrass fitness value 10 Machines vs. 1 Machine 20 Machines vs. 1 Machine

Supralinear Speedup !

47

Demo CPU vs. GPU (Weierstrass)

48

Demo 4 GPU island (Weierstrass)

49

50

Conclusion
!! Evolutionary Algorithms can be easily parallelized on GPU thanks to the EASEA language !! EASEA allows to generate a lot of different evolutionary paradigms !! EASEA is accessible to anyone with a bit of C knowledge

51