Neuroevolution & Evolutionary Algorithms Framework
Leonard Mosescu
Outline
Evolutionary Algorithms
Darwin Framework Overview
Design & Implementation
Evolutionary Algorithms
Evolutionary Algorithms Template
initialize_population
while(not satisfied):
for_each individual:
evaluate_fitness
create_next_generation:
select_parents
use crossover & mutation to generate children
Evolution loop
(one generation)
Example: Travelling Salesman Problem
A
D
E
B
C
C | B | D | E | A |
Solution Encoding
Example: Travelling Salesman Problem
A
D
E
B
C
C | B | D | E | A |
D | A | B | C | E |
Parent 1
Parent 2
Example: Travelling Salesman Problem
A
D
E
B
C
C | B | D | E | A |
A | B | D | E | C |
D | A | B | C | E |
Parent 1
Parent 2
Crossover
Example: Travelling Salesman Problem
C | B | D | E | A |
A | D | B | E | C |
A | B | D | E | C |
D | A | B | C | E |
Parent 1
Parent 2
Crossover
Mutation
A
D
E
B
C
Evolutionary Algorithms Applications
Artificial Neural Networks (ANNs)
An (Artificial) Neuron
Neural Networks Topologies
AI & Machine Learning
ML
AI
DeepLearning
* Not to scale
Evolutionary Algorithms +
Artificial Neural Networks = Neuroevolution
Neuroevolution
ML
AI
DeepLearning
* Not to scale
Evolutionary Algorithms
Neuroevolution
Darwin Framework Overview
Motivation
Key EA Abstractions
Populations and Domains
Evolution Loop
population->createPrimordialGeneration(population_size);
while (domain->evaluatePopulation(population)) {
population->rankGenotypes();
population->createNextGeneration();
}
Darwin Universe Database
Universe
Experiment
Variation
Trace
Darwin Universe Database
Built-in Domains
Built-in Populations
Conventional Neuroevolution (CNE)
Neuroevolution of augmenting topologies (NEAT)
Cartesian Genetic Programming (CGP)
Experiment Results: Visualizing Genotypes
NEAT Artificial Neural Network
Cartesian Genetic Programming
Experiment Results: Fitness
Darwin Studio
Design & Implementation
struct Config : public core::PropertySet {
PROPERTY(max_value, int, 100, "Maximum value");
PROPERTY(resolution, float, 0.3f, "Display resolution");
PROPERTY(name, string, "darwin", "Name");
PROPERTY(layers, vector<int>, {}, "Hidden layer sizes");
};
Config config;
// set Config::max_value
config.max_value = 75;
// read Config::name
auto name = config.name;
void printProperties(const core::PropertySet* config) {
// enumerate properties
for (const auto& property : config->properties()) {
// read the property name and value as strings
core::log("%s = %s\n", property->name(), property->value());
}
}
Defining properties
Direct member read / write
Runtime reflection
Using “cutting edge” C++
Error Handling
Generating Random Numbers
Cheap Dependencies Have a High Cost
Third Party Libraries In Darwin
References
QUESTIONS?
Bonus Slides
Evolutionary Algorithms Taxonomy
Why Evolutionary Algorithms?
Why not Evolutionary Algorithms?
Isn’t EA the same as Reinforcement Learning?
Timeline: Neuroevolution & Machine Learning
Darwin Framework Architecture
Darwin Framework: Core Pillars
Language Choice: C++
Cons:
Misc
Coding style
Tests
Documentation
Build System
How evolutionary selection can train more capable self-driving cars
“Now, Waymo, in a research collaboration with DeepMind, has taken inspiration from Darwin’s insights into evolution to make this training more effective and efficient.
...
Population Based Training (PBT) enabled dramatic improvements in model performance. ... A chief advantage of evolutionary methods such as PBT is that they can optimise arbitrarily complex metrics” (DeepMind and Waymo)
The gap between research and application
“Similar to what happened in Computer Vision, the progress in RL is not driven as much as you might reasonably assume by new amazing ideas. In Computer Vision, the 2012 AlexNet was mostly a scaled up (deeper and wider) version of 1990’s ConvNets. Similarly, the ATARI Deep Q Learning paper from 2013 is an implementation of a standard algorithm [...]” (Deep Reinforcement Learning: Pong from Pixels, May 2016)
The Law of Uphill Analysis and Downhill Synthesis
(Braitenberg's law)