1 of 53

Darwin

Neuroevolution & Evolutionary Algorithms Framework

Leonard Mosescu

2 of 53

Outline

Evolutionary Algorithms

Darwin Framework Overview

Design & Implementation

3 of 53

Evolutionary Algorithms

4 of 53

5 of 53

6 of 53

Evolutionary Algorithms Template

initialize_population

while(not satisfied):

for_each individual:

evaluate_fitness

create_next_generation:

select_parents

use crossover & mutation to generate children

Evolution loop

(one generation)

7 of 53

Example: Travelling Salesman Problem

A

D

E

B

C

C

B

D

E

A

Solution Encoding

8 of 53

Example: Travelling Salesman Problem

A

D

E

B

C

C

B

D

E

A

D

A

B

C

E

Parent 1

Parent 2

9 of 53

Example: Travelling Salesman Problem

A

D

E

B

C

C

B

D

E

A

A

B

D

E

C

D

A

B

C

E

Parent 1

Parent 2

Crossover

10 of 53

Example: Travelling Salesman Problem

C

B

D

E

A

A

D

B

E

C

A

B

D

E

C

D

A

B

C

E

Parent 1

Parent 2

Crossover

Mutation

A

D

E

B

C

11 of 53

Evolutionary Algorithms Applications

  • VLSI design (routing / placement)
  • Fuzzy testing
  • Floor plan design
  • Resource allocation
  • ...
  • Robotics
  • Artificial Life & AI

12 of 53

Artificial Neural Networks (ANNs)

13 of 53

An (Artificial) Neuron

14 of 53

Neural Networks Topologies

15 of 53

AI & Machine Learning

ML

AI

DeepLearning

* Not to scale

16 of 53

Evolutionary Algorithms +

Artificial Neural Networks = Neuroevolution

17 of 53

Neuroevolution

ML

AI

DeepLearning

* Not to scale

Evolutionary Algorithms

Neuroevolution

18 of 53

Darwin Framework Overview

19 of 53

Motivation

  • More time building fun stuff, less time on repetitive stuff
  • N domains x M algorithms -> N + M
  • Structured experimentation
  • Amortize the cost of tool building

20 of 53

Key EA Abstractions

  • Solution encoding (genotype)
  • Solution materialization (phenotype)
  • Population
  • Fitness Evaluation (domain)

21 of 53

Populations and Domains

22 of 53

Evolution Loop

population->createPrimordialGeneration(population_size);

while (domain->evaluatePopulation(population)) {

population->rankGenotypes();

population->createNextGeneration();

}

23 of 53

Darwin Universe Database

Universe

Experiment

Variation

Trace

24 of 53

Darwin Universe Database

25 of 53

Built-in Domains

26 of 53

Built-in Populations

Conventional Neuroevolution (CNE)

  • Feedforward
  • LSTM
  • RNN

Neuroevolution of augmenting topologies (NEAT)

Cartesian Genetic Programming (CGP)

27 of 53

Experiment Results: Visualizing Genotypes

NEAT Artificial Neural Network

Cartesian Genetic Programming

28 of 53

Experiment Results: Fitness

29 of 53

Darwin Studio

30 of 53

Design & Implementation

31 of 53

  • Basic reflection for structures
    • Needed for UI & serialization (and a bit of type erasure)
  • As cheap as a plain struct when accessing members
  • Built-in to/from JSON
    • fromJson() is not strict
  • Dedicated UI component: core_ui::PropertiesWidget
  • See also: core::PropertySetVariant

32 of 53

struct Config : public core::PropertySet {

PROPERTY(max_value, int, 100, "Maximum value");

PROPERTY(resolution, float, 0.3f, "Display resolution");

PROPERTY(name, string, "darwin", "Name");

PROPERTY(layers, vector<int>, {}, "Hidden layer sizes");

};

Config config;

// set Config::max_value

config.max_value = 75;

// read Config::name

auto name = config.name;

void printProperties(const core::PropertySet* config) {

// enumerate properties

for (const auto& property : config->properties()) {

// read the property name and value as strings

core::log("%s = %s\n", property->name(), property->value());

}

}

Defining properties

Direct member read / write

Runtime reflection

33 of 53

Using “cutting edge” C++

  • Coding style based on Chromium style (Google style)
    • … but with no restrictions
  • May use any feature which is supported across all the target platforms
    • Currently this is C++ 17
  • Examples
    • Function return type deduction [C++14]
    • Generic lambdas [C++14]
    • Lambda capture expressions [C++14]
    • [[maybe_unused]] [C++17]
    • Fold expressions [C++17]
    • Structured binding declarations [C++17]
    • std::filesystem (std::experimental::filesystem) [C++17]
    • std::optional [C++17]

34 of 53

Error Handling

  • Assert everything!
  • … and always check your assertions!
  • Fail fast!
  • C++ exceptions

35 of 53

Generating Random Numbers

  • C++11’s <random>
  • Seeding & Entropy
  • Generators
  • Performance

36 of 53

Cheap Dependencies Have a High Cost

  • Costs
    • Identify requirements up front
    • Research and evaluate options
    • Integration
    • Learn APIs
    • Implementation surprises
    • API seams
  • High bar for third party code
    • Compatible license
    • Platform support
    • Build system, test, support, documentation, dependencies, etc
    • API surface / functionality ratio.

37 of 53

Third Party Libraries In Darwin

  • Google-style third_party: dependencies are part of the source tree
  • Integration tests for third party libraries
  • No package manager
  • Git submodules
  • Start with a custom solution & shop for a mature library later

38 of 53

References

39 of 53

QUESTIONS?

40 of 53

Bonus Slides

41 of 53

Evolutionary Algorithms Taxonomy

42 of 53

Why Evolutionary Algorithms?

  • Simple, generic and reusable structure
  • Applicable to complex domains
    • No differentiable landscape requirement compared to Gradient Descent
  • Natural expression of the domain problem
  • Scales to huge search spaces
  • Potential for creativity & emergence
  • Can be resilient to local optima traps*

43 of 53

Why not Evolutionary Algorithms?

  • Outperformed by specialized algorithms
  • Designing genetic encodings and operators
  • Blackbox
  • Can be hard to debug and interpret solutions
  • Stochastic nature
  • Resilient to implementation mistakes

44 of 53

Isn’t EA the same as Reinforcement Learning?

  • Agent / Environment
  • Sparse Rewards
  • Model-based vs. Model-free RL
  • Timescales: Generation vs. Episode
  • DeepRL comes with the same DL limitations

45 of 53

Timeline: Neuroevolution & Machine Learning

46 of 53

Darwin Framework Architecture

47 of 53

Darwin Framework: Core Pillars

  • First class support for Evolutionary Algorithms concepts
  • Capable of running interesting experiments on easily available hardware
  • Complete package
  • Structured approach to experimentation
  • Cross-platform with minimal external dependencies

48 of 53

Language Choice: C++

  • Mature cross-platform tooling
  • Reduced dependencies / layers
  • Qt is still one of the best desktop GUI toolkits
  • Good selection of native, 3rd party libraries
  • C++ provides one of the best resource management solutions

Cons:

  • Still missing basic features: reflection, run-time code generation
  • Poor build model
  • Too much flexibility in the wrong places (lack of universal conventions)
  • Complexity / cognitive load

49 of 53

Misc

Coding style

  • Adopted a mature, well documented style (Chromium / Google style)
  • Use clang-format!

Tests

Documentation

  • Doxygen is still the most flexible documentation tool for C++
  • github.io is a great documentation hosting platform

Build System

50 of 53

How evolutionary selection can train more capable self-driving cars

Now, Waymo, in a research collaboration with DeepMind, has taken inspiration from Darwin’s insights into evolution to make this training more effective and efficient.

...

Population Based Training (PBT) enabled dramatic improvements in model performance. ... A chief advantage of evolutionary methods such as PBT is that they can optimise arbitrarily complex metrics” (DeepMind and Waymo)

51 of 53

The gap between research and application

Similar to what happened in Computer Vision, the progress in RL is not driven as much as you might reasonably assume by new amazing ideas. In Computer Vision, the 2012 AlexNet was mostly a scaled up (deeper and wider) version of 1990’s ConvNets. Similarly, the ATARI Deep Q Learning paper from 2013 is an implementation of a standard algorithm [...]” (Deep Reinforcement Learning: Pong from Pixels, May 2016)

52 of 53

The Law of Uphill Analysis and Downhill Synthesis

(Braitenberg's law)

53 of 53