1 of 245

Applying Wafer-Scale Evolutionary Simulations to Investigate Hypermutator Dynamics in Large Asexual Populations

(and phylogeny tracking, too)

February 21, 2025 @ BEACON Seminar

Matthew Andres Moreno

Complex Systems, Michigan Institute for Data and AI in Society, Ecology and Evolutionary Biology

University of Michigan

2 of 245

Agent-based Evolution & Simulation Scale

📚 [slides, paper, code] https://hopth.ru/de

3 of 245

Scale and Agent-based Evolution Simulations

📧 morenoma@umich.edu

(Dolson and Ofria, 2021)

4 of 245

Scale and Agent-based Evolution Simulations

📧 morenoma@umich.edu

(Dolson and Ofria, 2021)

5 of 245

Scale and Digital Evolution

Avida

(Ofria and Wilke, 2009)

📧 morenoma@umich.edu

6 of 245

Evolution Models in vivo and in silico

LTEE

(Good et al., 2017)

📧 morenoma@umich.edu

E.

coli

experimental evolution:

n>1, experimental manipulations

7 of 245

Evolution Models in vivo and in silico

LTEE

(Good et al., 2017)

Avida

(Ofria and Wilke, 2009)

📧 morenoma@umich.edu

E.

coli

experimental evolution:

n>1, experimental manipulations

simulation and modeling:

synthesize theory, test sufficiency

8 of 245

Evolution Models in vivo and in silico

LTEE

(Good et al., 2017)

Avida

(Ofria and Wilke, 2009)

📧 morenoma@umich.edu

E.

coli

9 of 245

Scale and Digital Evolution

10 of 245

1

processor

~billion replications/day

11 of 245

1

processor

12 of 245

1

processor

13 of 245

Grace Hopper

14 of 245

Cgoodwin, CC BY-SA 4.0 <https://creativecommons.org/licenses/by-sa/4.0>, via Wikimedia Commons

Nejones1987, CC BY-SA 4.0 <https://creativecommons.org/licenses/by-sa/4.0>, via Wikimedia Commons

a bigger ox

a team of oxen

✓

Grace Hopper

15 of 245

Cgoodwin, CC BY-SA 4.0 <https://creativecommons.org/licenses/by-sa/4.0>, via Wikimedia Commons

Nejones1987, CC BY-SA 4.0 <https://creativecommons.org/licenses/by-sa/4.0>, via Wikimedia Commons

a bigger ox

a team of oxen

✓

Grace Hopper

16 of 245

KLAT2 (HANK DIETZ)

17 of 245

KLAT2 (HANK DIETZ)

18 of 245

KLAT2 (HANK DIETZ)

Fugaku

19 of 245

KLAT2 (HANK DIETZ)

The Library of Congress via Wikimedia Commons

“next-generation” high-performance computing (especially AI/ML)

20 of 245

KLAT2 (HANK DIETZ)

The Library of Congress via Wikimedia Commons

NVIDIA A100 GPU

6,912 CUDA cores

21 of 245

KLAT2 (HANK DIETZ)

The Library of Congress via Wikimedia Commons

Graphcore IPU

1,200 cores per chip

clustered up to 1,024

chips

NVIDIA A100 GPU

6,912 CUDA cores

SambaNova

RDU

22 of 245

KLAT2 (HANK DIETZ)

Cerebras

Wafer-Scale

Engine

The Library of Congress via Wikimedia Commons

850,000

cores

NVIDIA A100 GPU

6,912 CUDA cores

Graphcore IPU

1,200 cores per chip

clustered up to 1,024

chips

SambaNova

RDU

23 of 245

…

Cerebras Wafer-Scale Engine

850,000

cores

📧 morenoma@umich.edu

📚 [these slides] https://hopth.ru/ea

24 of 245

…

Cerebras Wafer-Scale Engine

850,000

cores

📧 morenoma@umich.edu

📚 [these slides] https://hopth.ru/ea

25 of 245

…

Cerebras Wafer-Scale Engine

850,000

cores

📧 morenoma@umich.edu

📚 [these slides] https://hopth.ru/ea

26 of 245

…

Cerebras Wafer-Scale Engine

850,000

cores

📧 morenoma@umich.edu

📚 [these slides] https://hopth.ru/ea

27 of 245

…

Cerebras Wafer-Scale Engine

850,000

cores

📧 morenoma@umich.edu

📚 [these slides] https://hopth.ru/ea

data

output

28 of 245

…

Cerebras Wafer-Scale Engine

850,000

cores

📧 morenoma@umich.edu

📚 [these slides] https://hopth.ru/ea

only ~48kb

mem per core

29 of 245

ByteBoost

Workshop ‘24

at supercomputing

30 of 245

ByteBoost

Workshop ‘24

31 of 245

ByteBoost

Workshop ‘24

32 of 245

ByteBoost

Workshop ‘24

33 of 245

GraphCore IPU — another AI/ML accelerator

1,200 cores per chip

clustered up to 1,024 chips

34 of 245

Goal: develop and apply methods to harness next-generation high-performance computing hardware to enable larger agent-based evolution simulations

35 of 245

Part 1:

Hypermutator Dynamics Experiments

on the Wafer-scale Engine

Goal: develop and apply methods to harness next-generation high-performance computing hardware to enable larger agent-based evolution simulations

36 of 245

Part 1:

Hypermutator Dynamics Experiments

on the Wafer-scale Engine

Part 2:

Decentralized Phylogeny Tracking

Goal: develop and apply methods to harness next-generation high-performance computing hardware to enable larger agent-based evolution simulations

37 of 245

Goal: develop methods to harness next-generation high-performance computing hardware to enable larger agent-based evolution simulations

38 of 245

Develop Algorithms and Software for Wafer-Scale Evolution Simulations

Technical Proof-of-Concept Runs

Proof-of

-Concept

Hypothesis-driven Experiments

Goal: develop methods to harness next-generation high-performance computing hardware to enable larger agent-based evolution simulations

we are here

39 of 245

Develop Algorithms and Software for Wafer-Scale Evolution Simulations

Technical Proof-of-Concept Runs

Proof-of

-Concept

Hypothesis-driven Experiments

we are here

Goal: develop methods to harness next-generation high-performance computing hardware to enable larger agent-based evolution simulations

hypermutator dynamics experiments

II. phylogeny

tracking

40 of 245

Part 1:

Hypermutator Dynamics Experiments

on the Wafer-scale Engine

41 of 245

📧 morenoma@umich.edu

42 of 245

“normomutator”

“hypermutator”

43 of 245

hypermutator trait

👿 –

higher

deleterious

mutation

load

😇 +

faster

discovery

beneficial

mutations

44 of 245

“hypermutator”

https://mathematical-oncology.org/blog/evofreq-and-hal.html

45 of 245

“hypermutator”

https://mathematical-oncology.org/blog/evofreq-and-hal.html

46 of 245

“hypermutator”

https://mathematical-oncology.org/blog/evofreq-and-hal.html

47 of 245

Hypermutator dynamics pertain to

pathogen evolution
antibiotic resistance
tumor cell evolution
etc.

📧 morenoma@umich.edu

📚 [these slides] https://hopth.ru/ea

48 of 245

hypermutator selection — Raynes et al., 2018

📧 morenoma@umich.edu

📚 [these slides] https://hopth.ru/ea

49 of 245

hypermutator selection — Raynes et al., 2018

📧 morenoma@umich.edu

📚 [these slides] https://hopth.ru/ea

hypermutators favored

normomutators favored

50 of 245

hypermutator selection — Raynes et al., 2018

📧 morenoma@umich.edu

📚 [these slides] https://hopth.ru/ea

hyper

mutator

hyper

mutator

hyper

mutator

hyper

mutator

hyper

mutator

hyper

mutator

hyper

mutator

hyper

mutator

hyper

mutator

hyper

mutator

hyper

mutator

hyper

mutator

hyper

mutator

hyper

mutator

hyper

mutator

hyper

mutator

51 of 245

📚 [these slides] https://hopth.ru/ea

hypermutator selection — Raynes et al., 2018

hypermutators win

<— pop size 10k

52 of 245

?????????

📚 [these slides] https://hopth.ru/ea

hypermutator selection — Raynes et al., 2018

very large

population sizes??

53 of 245

agent-based model

beneficial

λ=1 in 100k

deleterious

λ=1 in 1k

poisson distribution

hypermutator selection — Raynes et al., 2018

54 of 245

beneficial

λ=1 in 1k

deleterious

λ=1 in 10

hypermutator

100x mutation rate

poisson distribution

hypermutator selection — Raynes et al., 2018

agent-based model

55 of 245

←hypermutator

←normomutator

hypermutator selection — Raynes et al., 2018

50/50

56 of 245

←hypermutator

Generations

←normomutator

hypermutator selection — Raynes et al., 2018

50/50

57 of 245

←hypermutator

Generations

←normomutator

hypermutator selection — Raynes et al., 2018

50/50

— versus —

hypermuts win

normomuts win

58 of 245

Island Model GA

Treatment Conditions. Allowed mutational outcomes under compared outcomes. Note that adaptive regime introduces the possibility for selective sweeps.

a) purifying

regime

On-device Performance

b) adaptive

regime

Example reconstructions. Phylogenetic reconstructions of 1 million generation on-hardware simulations. For legibility, phylogeny visualizations were further subsampled to 1k end-state agents. Left phylogenies are log-scaled with ultrametric correction to better show topology and right phylogenies are linear-scaled.

On-device Evolution Trial

Phylometric outcomes. Statistics calculated from reconstruction of 18 million population size/1 million generation on-hardware simulations. Phylometrics were calculated from reconstructions with 10k sampled end-state agents.

🏝️ 🏝️ 🏝️ 🏝️

“

”

📧 morenoma@umich.edu

🏝️ 🏝️ 🏝️ 🏝️

59 of 245

Island Model GA

Treatment Conditions. Allowed mutational outcomes under compared outcomes. Note that adaptive regime introduces the possibility for selective sweeps.

a) purifying

regime

On-device Performance

b) adaptive

regime

Example reconstructions. Phylogenetic reconstructions of 1 million generation on-hardware simulations. For legibility, phylogeny visualizations were further subsampled to 1k end-state agents. Left phylogenies are log-scaled with ultrametric correction to better show topology and right phylogenies are linear-scaled.

On-device Evolution Trial

Phylometric outcomes. Statistics calculated from reconstruction of 18 million population size/1 million generation on-hardware simulations. Phylometrics were calculated from reconstructions with 10k sampled end-state agents.

🏝️ 🏝️ 🏝️ 🏝️

“

”

📧 morenoma@umich.edu

60 of 245

Simulation Results

61 of 245

Preliminary Results

62 of 245

Preliminary Results

normo

lose

63 of 245

Preliminary Results

64 of 245

Preliminary Results

65 of 245

Preliminary Results

normo win in big pops when few beneficial mutations are available

66 of 245

Preliminary Results

normo win in big pops when few beneficial mutations are available

67 of 245

Preliminary Results

68 of 245

Preliminary Results

3 beneficial

available

normo

lose in large pops

pop size up to ~10 million

69 of 245

Preliminary Results

normo

win in large pops

5 beneficial

available

+very large pop size

pop size up to ~1.5 billion

70 of 245

Generations

50/50

71 of 245

50/50

Generations

1 in 100k

denovo

Generations

72 of 245

normomutator

hypermutator

hypermutators fix

normomutators persist

https://hopth.ru/ej

https://hopth.ru/ek

net population size

191 million

num generations

200,000

real time

~70 seconds

73 of 245

hypermutators fix

normomutator

hypermutator

https://hopth.ru/ej

https://hopth.ru/ek

net population size

191 million

num generations

200,000

real time

~70 seconds

normomutators persist

74 of 245

hypermutators fix

normomutators persist

normomutator

hypermutator

https://hopth.ru/ej

https://hopth.ru/ek

net population size

191 million

num generations

200,000

real time

~70 seconds

75 of 245

Preliminary Results

+denovo hypermutator origination

76 of 245

+denovo hypermutator origination

5 → 14 beneficial muts available

normo

win

in large pops

77 of 245

well-mixed

1D ring

2D grid

on GPU

(stronger) spatial structure (weaker)

spatial structure

(✂️ for time)

78 of 245

well-mixed

1D ring

2D grid

on GPU

(stronger) spatial structure (weaker)

normomutators more resilient

normomutators less resilient

79 of 245

well-mixed

1D ring

2D grid

on GPU

(stronger) spatial structure (weaker)

80 of 245

on GPU

well-mixed

1D ring

2D grid

normomutators more resilient

81 of 245

on GPU

well-mixed

1D ring

2D grid

normomutators

more resilient

(>20 ben muts avail)

(stronger) spatial structure (weaker)

82 of 245

on GPU

well-mixed

1D ring

2D grid

normomutators

more resilient

(>20 ben muts avail)

normomutators

less resilient

(<8 ben muts avail)

(stronger) spatial structure (weaker)

83 of 245

on GPU

well-mixed

1D ring

2D grid

normomutators more resilient

normomutators less resilient

84 of 245

Performance 🏎️

85 of 245

Performance

vs. A100 GPU (CuPy)

Speedup

~ 294x faster

WSE vs GPU

WSE ~8x10¹¹(800 billion) agent-generations per second

86 of 245

WSE ~8x10¹¹(800 billion) agent-generations per second

87 of 245

WSE ~8x10¹¹(800 billion) agent-generations per second

295× vs GPU

88 of 245

Performance

Agents per PE	Net Population Size	Simulation Speed (Generations per Second)

89 of 245

Performance

Agents per PE	Net Population Size	Simulation Speed (Generations per Second)
256	190.8 million	4,306 (std 156)

90 of 245

Performance

Agents per PE	Net Population Size	Simulation Speed (Generations per Second)
256	190.8 million	4,306 (std 156)
2048	1.5 billion	549 (std 1)

91 of 245

Performance

WSE ~8x10¹¹(800 billion) agent-generations per second

92 of 245

Performance

295× vs GPU

93 of 245

Performance

295× vs GPU

111,091× vs CPU

94 of 245

Performance

all ~8x10¹¹(800 billion) agent-generations per second

Agents per PE	Net Population Size	Simulation Speed (Generations per Second)
256	190.8 million	4,306 (std 156)
2048	1.5 billion	549 (std 1)

95 of 245

Conclusion

96 of 245

Summary

Under unlimited adaptive potential, large asexual populations favor hypermutators (Raynes et al., 2018)

97 of 245

Summary

Under unlimited adaptive potential, large asexual populations favor hypermutators (Raynes et al., 2018)
Under restricted adaptive potential, normomutators regain favor in very large populations

98 of 245

Summary

Under unlimited adaptive potential, large asexual populations favor hypermutators (Raynes et al., 2018)
Under restricted adaptive potential, normomutators regain favor in very large populations

This effect is amplified when hypermutators are initially rare, with strong spatial structure

99 of 245

Next Steps

more sophisticated fitness landscape models
hypermutator reversion
connect to in vivo data?

100 of 245

Next Steps

VS

osmotic pressure

more adaptive loci

antibiotics

fewer adaptive loci

Martjin Callens et al., 2023

more sophisticated fitness landscape models
hypermutator reversion
connect to in vivo data?

E. Coli

101 of 245

Next Steps

VS

osmotic pressure

more adaptive loci

→

more hypermutators

antibiotics

fewer adaptive loci

→

fewer hypermutators

Martjin Callens et al., 2023

more sophisticated fitness landscape models
hypermutator reversion
connect to in vivo data?

E. Coli

102 of 245

Part II:

Decentralized Phylogeny Tracking

📧 morenoma@umich.edu

103 of 245

…

Cerebras Wafer-Scale Engine

48kb mem

😶‍🌫️

📧 morenoma@umich.edu

104 of 245

…

Cerebras Wafer-Scale Engine

48kb mem

😶‍🌫️

🌸

🌺

🌼

🌻

phylogeny

📧 morenoma@umich.edu

105 of 245

Perfect Tracking:

Complete Observability

106 of 245

Perfect Tracking:

Complete Observability

(Nozoe et al, 2017)

🦠

🔬

🦠

107 of 245

Perfect Tracking:

Complete Observability

(Nozoe et al, 2017)

🔬

🦠

108 of 245

Perfect Tracking:

Complete Observability

🌸

🌺

🌼

🌻

🦠

👍

(Nozoe et al, 2017)

🔬

🦠

109 of 245

Perfect Tracking:

Complete Observability

🌸

🌺

🌼

🌻

👍

(Nozoe et al, 2017)

🔬

👎

🦠

110 of 245

Perfect Tracking:

Complete Observability

🌸

🌺

🌼

🌻

👍

👎

(Nozoe et al, 2017)

🔬

🦠

111 of 245

Highly-distributed,

Many-core Computation

@MorenoMatthewA 🐘@mas.to

🌸

🌺

🌼

🌻

🚫

❓

112 of 245

@MorenoMatthewA 🐘@mas.to

🌻

🔬

🦠

113 of 245

@MorenoMatthewA 🐘@mas.to

🌸

🌺

🌼

🌻

🔬

🦠

114 of 245

@MorenoMatthewA 🐘@mas.to

🌸

🌺

🌼

🌻

🧬

🌻

🔬

🦠

115 of 245

@MorenoMatthewA 🐘@mas.to

🪄🐇

🌸

🌺

🌼

🌻

🌺

🌼

🪷

🧬

🌻

🔬

🦠

116 of 245

@MorenoMatthewA 🐘@mas.to

🌸

🌺

🌼

🌻

🌸

🌺

🌼

🪷

🧬

*inferred estimate

🧬

🌻

🔬

🦠

117 of 245

@MorenoMatthewA 🐘@mas.to

🌸

🌺

🌼

🌻

🌸

🌺

🌼

🪷

🧬

*inferred estimate

How to design

.🧬 to facilitate reconstruction?

118 of 245

@MorenoMatthewA 🐘@mas.to

🌸

🌺

🌼

🌻

🌸

🌺

🌼

🪷

🧬

*inferred estimate

Hwang et al., 2019

Crispr-cas9 Barcoding

119 of 245

@MorenoMatthewA 🐘@mas.to

🌸

🌺

🌼

🌻

🌸

🌺

🌼

🪷

🧬

…

🧬

= “hstrat” annotation

120 of 245

@MorenoMatthewA 🐘@mas.to

🌸

🌺

🌼

🌻

🌸

🌺

🌼

🪷

🧬

= “hstrat” annotation

…

🧬

small (~96 bits)
fast (<100ns CPU)
tunably accurate

121 of 245

evolve

under-the-hood “hstrat” algorithm

(✂️ for time)

122 of 245

evolve

123 of 245

evolve

124 of 245

evolve

125 of 245

.B.

.C.

.A.

evolve

126 of 245

.B.

.C.

.A.

evolve

127 of 245

.C.

.B.

.A.

.B.

.C.

.A.

evolve

end-state

128 of 245

.C.

.B.

.A.

.C.

.B.

.A.

.B.

.C.

.A.

evolve

end-state

reconstruct

129 of 245

t=8

130 of 245

t=8

131 of 245

t=8

t=16

132 of 245

t=8

t=16

t=8

t=16

133 of 245

t=8

t=16

t=8

t=16

“steady”

“tilted”

134 of 245

t=8

t=16

t=8

t=16

“steady”

“tilted”

This is the hard/interesting part!!!

135 of 245

136 of 245

137 of 245

Problem Setup — [props?]

138 of 245

Problem Setup — [props?]

139 of 245

Proof-of-concept Experiment (Alife ‘24)

Treatment Conditions. Allowed mutational outcomes under compared outcomes. Note that adaptive regime introduces the possibility for selective sweeps.

a) purifying

regime

b) adaptive

regime

Example reconstructions. Phylogenetic reconstructions of 1 million generation on-hardware simulations. For legibility, phylogeny visualizations were further subsampled to 1k end-state agents. Left phylogenies are log-scaled with ultrametric correction to better show topology and right phylogenies are linear-scaled.

On-device Evolution Trial

Phylometric outcomes. Statistics calculated from reconstruction of 18 million population size/1 million generation on-hardware simulations. Phylometrics were calculated from reconstructions with 10k sampled end-state agents.

🏝️ 🏝️ 🏝️ 🏝️

“

”

140 of 245

Proof-of-concept Experiment (Alife ‘24)

Treatment Conditions. Allowed mutational outcomes under compared outcomes. Note that adaptive regime introduces the possibility for selective sweeps.

a) purifying

regime

b) adaptive

regime

Example reconstructions. Phylogenetic reconstructions of 1 million generation on-hardware simulations. For legibility, phylogeny visualizations were further subsampled to 1k end-state agents. Left phylogenies are log-scaled with ultrametric correction to better show topology and right phylogenies are linear-scaled.

a) purifying regime

b) adaptive regime

👾 =mutate⇒

👾 or 👾^—

👾 =mutate⇒

👾 or 👾^— or 👾⁺

On-device Evolution Trial

Phylometric outcomes. Statistics calculated from reconstruction of 18 million population size/1 million generation on-hardware simulations. Phylometrics were calculated from reconstructions with 10k sampled end-state agents.

🏝️ 🏝️ 🏝️ 🏝️

“

”

simple agent model, explicit fitness

141 of 245

Proof-of-concept Experiment (Alife ‘24)

🧬

Treatment Conditions. Allowed mutational outcomes under compared outcomes. Note that adaptive regime introduces the possibility for selective sweeps.

a) purifying

regime

b) adaptive

regime

Example reconstructions. Phylogenetic reconstructions of 1 million generation on-hardware simulations. For legibility, phylogeny visualizations were further subsampled to 1k end-state agents. Left phylogenies are log-scaled with ultrametric correction to better show topology and right phylogenies are linear-scaled.

a) purifying regime

b) adaptive regime

👾 =mutate⇒

👾 or 👾^—

👾 =mutate⇒

👾 or 👾^— or 👾⁺

On-device Evolution Trial

Phylometric outcomes. Statistics calculated from reconstruction of 18 million population size/1 million generation on-hardware simulations. Phylometrics were calculated from reconstructions with 10k sampled end-state agents.

🏝️ 🏝️ 🏝️ 🏝️

“

”

simple agent model, explicit fitness

🧬

142 of 245

Example Phylogenies

a) purifying

regime

b) adaptive

regime

Phylometric outcomes. Statistics calculated from reconstruction of 18 million population size/1 million generation on-hardware simulations. Phylometrics were calculated from reconstructions with 10k sampled end-state agents.

📧 morenoma@umich.edu

~ 8 million agents

~ 10k tip phylogenies

(-) adaptive muts

(+) adaptive muts

143 of 245

Evolutionary Inference from Phylogeny Structure

📚 [slides, paper, code] https://hopth.ru/de

144 of 245

Phylogeny Structure Metrics

📚 [slides, paper, code] https://hopth.ru/de

145 of 245

Evolutionary Inference from Phylogeny Structure

📚 [slides, paper, code] https://hopth.ru/de

146 of 245

Goal: end-to-end encapsulated workflow

raw sim “genome” strings

🧬🧬🧬🧬🧬🧬🧬

🌻

🌸

🌺

🌼

🌻

phylogeny

One Terminal

Command

147 of 245

Goal: end-to-end encapsulated workflow

raw sim “genome” strings

🧬🧬🧬🧬🧬🧬🧬

2. extract and decode

markers

3. build tree

🌻

🌸

🌺

🌼

🌻

4. phylogeny

courtesy meat-machinery.com

148 of 245

Goal: end-to-end encapsulated workflow

raw sim “genome” strings

🧬🧬🧬🧬🧬🧬🧬

2. extract and decode

markers

3. build tree

🌻

🌸

🌺

🌼

🌻

4. phylogeny

courtesy meat-machinery.com

149 of 245

Goal: end-to-end encapsulated workflow

courtesy meat-machinery.com

raw sim “genome” strings

🧬🧬🧬🧬🧬🧬🧬

2. extract and decode

markers

3. build tree

🌻

🌸

🌺

🌼

🌻

4. phylogeny

Joey

Wagner

⏱️

Connor Yang

Vivaan Singhvi

150 of 245

Goal: high-performance, easy phylogeny reconstruction workflow

raw sim “genome” strings

🧬🧬🧬🧬🧬🧬🧬

🌻

🌸

🌺

🌻

phylogeny

Joey

Wagner

Connor

Yang

Vivaan Singhvi

reconstruction algorithm

151 of 245

Goal: high-performance, easy phylogeny reconstruction workflow

throughput:

4.7-6.6 million tips / minute
1 billion tips ~ 2.5 to 3.5 hours

raw sim “genome” strings

🧬🧬🧬🧬🧬🧬🧬

🌻

🌸

🌺

🌼

🌻

phylogeny

reconstruction algorithm

Joey

Wagner

Connor

Yang

Vivaan Singhvi

152 of 245

how to use?

Goal: high-performance, easy phylogeny reconstruction workflow

raw sim “genome” strings

🧬🧬🧬🧬🧬🧬🧬

🌻

🌸

🌺

🌼

🌻

phylogeny

reconstruction algorithm

153 of 245

🧬

how to annotate?

154 of 245

🧬

1.fixed-width bit field

2. generation counter

(e.g., 96 bits)

155 of 245

🧬

1.fixed-width bit field

2. generation counter

(e.g., 96 bits)

generation 572

156 of 245

🧬

1.fixed-width bit field

2. generation counter

(e.g., 96 bits)

DSTREAM LIBRARY

generation 572

stateless!

157 of 245

🧬

1.fixed-width bit field

2. generation counter

(e.g., 96 bits)

DSTREAM LIBRARY

generation 572

randomize bit 35

stateless!

158 of 245

🧬

1.fixed-width bit field

2. generation counter

(e.g., 96 bits)

DSTREAM LIBRARY

generation 572

randomize bit 35

stateless!

Python

C++�Rust

Zig

CSL

(or ~50 LOC)

159 of 245

how to reconstruct?

160 of 245

.csv, .parquet, etc.

how to reconstruct?

161 of 245

.csv, .parquet, etc.

83a3bc

genome hex

472a70

how to reconstruct?

162 of 245

.csv, .parquet, etc.

genome hex

83a3bc

hstrat bitfield offset & width

8, 32

472a70

8, 32

how to reconstruct?

163 of 245

✅ prototype → 🔜 open-source package

.csv, .parquet, etc.

83a3bc

8, 32

40, 16

hstrat bitfield offset & width

generation count offset & width

genome hex

472a70

8, 32

40, 16

how to reconstruct?

164 of 245

✅ prototype → 🔜 open-source package

83a3bc

8, 32

40, 16

(or command line flags)

hstrat bitfield offset & width

generation count offset & width

genome hex

472a70

8, 32

40, 16

how to reconstruct?

.csv, .parquet, etc.

165 of 245

1 Terminal

Command

.csv, .parquet, etc.

83a3bc

8, 32

40, 16

(or command line flags)

hstrat bitfield offset & width

generation count offset & width

genome hex

472a70

8, 32

40, 16

166 of 245

🌻

🌸

🌺

🌻

phylogeny

1 Terminal

Command

.csv, .parquet, etc.

83a3bc

8, 32

40, 16

(or command line flags)

hstrat bitfield offset & width

generation count offset & width

genome hex

472a70

8, 32

40, 16

.csv, .parquet, etc.

167 of 245

🌻

🌸

🌺

🌻

phylogeny

1 Terminal

Command

.csv, .parquet, etc.

83a3bc

8, 32

40, 16

(or command line flags)

via pip or singularity

hstrat bitfield offset & width

generation count offset & width

genome hex

472a70

8, 32

40, 16

.csv, .parquet, etc.

168 of 245

🌻

🌸

🌺

🌻

phylogeny

1 Terminal

Command

.csv, .parquet, etc.

my very cool phenotype data

via pip or singularity

83a3bc

8, 32

40, 16

hstrat bitfield offset & width

generation count offset & width

genome hex

472a70

8, 32

40, 16

.csv, .parquet, etc.

169 of 245

phylogeny

1 Terminal

Command

✅ prototype → 🔜 open-source package

.csv, .parquet, etc.

genome hex 🧬

83a3bc

data bit offset & width

8, 32

generation counter offset & width

40, 16

my very cool phenotype data

via pip or singularity

.csv, .parquet, etc.

170 of 245

high-throughput phylogeny generation (in vivo & in silico)

Liu et al., 2024:

235 million seqs

39k jobs, 30.5 hrs

🌸

🌺

🌼

🌻

171 of 245

high-throughput phylogeny generation (in vivo & in silico)

Liu et al., 2024:

235 million seqs

39k jobs, 30.5 hrs

🌸

🌺

🌼

🌻

existing phylogeny analyses and tools

172 of 245

existing phylogeny analyses and tools

what do you do with a billion tip phylogeny?????

(or 1,000 million tip phylogenies???)

173 of 245

what do you do with a billion tip phylogeny?????

(or 1,000 million tip phylogenies???)

174 of 245

high-throughput phylogeny generation (in vivo & in silico)

175 of 245

contemporary analyses

high-throughput phylogeny generation (in vivo & in silico)

Liu et al., 2024: 235 million seqs

39k jobs, 30.5 hrs

176 of 245

use more

phylogenies

use larger

phylogenies

contemporary analyses

need:

new stats/analyses
high-perf analysis code (TreeSwift, CompactTree, alifestd, etc)

high-throughput phylogeny generation (in vivo & in silico)

Liu et al., 2024: 235 million seqs

39k jobs, 30.5 hrs

177 of 245

Conclusion

178 of 245

“perfect” observability

(Nozoe et al,

2017)

🦠

179 of 245

“sampling-based” observability

“perfect” observability

(Nozoe et al,

2017)

🦠

🌸

🌺

🌼

🌻

🧬

🦠

180 of 245

“sampling-based” observability

“perfect” observability

(Nozoe et al,

2017)

🦠

🌸

🌺

🌼

🌻

🧬

🦠

in vivo

181 of 245

“sampling-based” observability

“perfect” observability

(Nozoe et al,

2017)

🦠

🌸

🌺

🌼

🌻

🧬

🦠

in silico

in vivo

182 of 245

“sampling-based” observability

“perfect” observability

(Nozoe et al,

2017)

🦠

🌸

🌺

🌼

🌻

🧬

🦠

🔬

… ← in vivo

in silico

183 of 245

“sampling-based” observability

“perfect” observability

(Nozoe et al,

2017)

🦠

🌸

🌺

🌼

🌻

🧬

🦠

🔬

in silico → …

… ← in vivo

184 of 245

“sampling-based” observability

“perfect” observability

(Nozoe et al,

2017)

🦠

🌸

🌺

🌼

🌻

🧬

🦠

🔬

in silico → …

… ← in vivo

185 of 245

“approximate” observability

“perfect” observability

(Nozoe et al,

2017)

🦠

🌸

🌺

🌼

🌻

🧬

🦠

in silico → …

… ← in vivo

🔬

………………………………………………………………………

186 of 245

(Dolson and Ofria, 2021)

187 of 245

Parallel/distributed Scale-up: Interesting Trade-offs

complete observability / sampling-based observability
perfect reproducibility / automated, well-specified protocols
machine independence / measurable machine influence
“real-time” agent execution affects fitness
experiments on PC / experiments require non-trivial infrastructure

(Dolson and Ofria, 2021)

188 of 245

Parallel/distributed Scale-up: Interesting Trade-offs

complete observability / sampling-based observability

(Dolson and Ofria, 2021)

189 of 245

PSC

Neocortex

(NSF ACCESS)

Argonne

Leadership

Compute Facility

(Dept of Energy)

Today

190 of 245

Condor Galaxy 3,4,5

PSC

Neocortex

(NSF ACCESS)

Argonne

Leadership

Compute Facility

(Dept of Energy)

64 CS-3 chips (54 million cores)�8 exaFLOPs

Today

Today ($$$)

191 of 245

Condor Galaxy 3,4,5

PSC

Neocortex

(NSF ACCESS)

Argonne

Leadership

Compute Facility

(Dept of Energy)

???

64 CS-3 chips (54 million cores)�8 exaFLOPs

As many as 2,048 systems can be combined…

1.84 billion cores

Today

Future

https://spectrum.ieee.org/cerebras-chip-cs3

Today ($$$)

192 of 245

Peter J. Park, CC BY 2.5 <https://creativecommons.org/licenses/by/2.5>, via Wikimedia Commons

kqedquest Via Flickr

193 of 245

ByteBoost

Workshop ‘24

194 of 245

Dr. Luis Zaman

@MorenoMatthewA 🐘@mas.to

This project is supported by the Eric and Wendy Schmidt AI in Science Postdoctoral Fellowship, a Schmidt Sciences program.

Dr. Emily Dolson

📧 morenoma@umich.edu

Joey Wagner

Connor Yang (UROP)

Vivaan Singhvi (UROP)

Office of Advanced Scientific Computing Research (ASCR)

Award Number DE-SC0025634

ByteBoost

Workshop ‘24

195 of 245

Questions?

We have a very bad tendency to base our plans for computers on the equipment we have in house and the things we’re doing now. And totally fail to review them in the light of the equipment that will be available and the things that we will be doing — I think the saddest phrase I ever hear in a computer installation is that horrible one "but we’ve always done it that way." That’s a forbidden phrase in my office.

Capt. Grace Hopper

Future Possibilities: Data, Hardware, Software, and People — August 26, 1982

196 of 245

References

Ackley, David H. "A Robust Programmable Replicator for an Indefinitely Scalable Machine." ALIFE 2023: Ghost in the Machine: Proceedings of the 2023 Artificial Life Conference. MIT Press, 2023.
Liquidware. Introducing the Illuminato X Machina. http://antipastohw.blogspot.com/2009/08/introducing-illuminato-x-machina.html
ALIEN Artificial Life Environment. https://www.alien-project.org/index.html
Dolson, Emily, and Charles Ofria. "Digital evolution for ecology research: a review." Frontiers in Ecology and Evolution 9 (2021): 750779.
Good, Benjamin H., et al. "The dynamics of molecular evolution over 60,000 generations." Nature 551.7678 (2017): 45-50.
Ofria, Charles, and Claus O. Wilke. "Avida: A software platform for research in computational evolutionary biology." Artificial life 10.2 (2004): 191-229.
Buitrago P.A., Nystrom N.A. (2021) Neocortex and Bridges-2: A High Performance AI+HPC Ecosystem for Science, Discovery, and Societal Good. In: Nesmachnow S., Castro H., Tchernykh A. (eds) High Performance Computing. CARLA 2020. Communications in Computer and Information Science, vol 1327. Springer, Cham. https://doi.org/10.1007/978-3-030-68035-0_15

197 of 245

ByteBoost Workshop ‘24

Neocortex @

Leighton Wilson and Mathias Jacquelin

@ Cerebras

198 of 245

Future Work

199 of 245

Results

200 of 245

agent-based model

beneficial

λ=1 in 100k

deleterious

λ=1 in 1k

poisson distribution

201 of 245

agent-based model

🏝️ 🏝️ 🏝️ 🏝️

202 of 245

203 of 245

1 billion agents

204 of 245

agent-based model

beneficial

λ=1 in 100k

deleterious

λ=1 in 1k

∞

205 of 245

vs

agent-based model

beneficial

λ=1 in 100k

deleterious

λ=1 in 1k

poisson distribution

∞

206 of 245

vs

agent-based model

beneficial

λ=1 in 100k

deleterious

λ=1 in 1k

poisson distribution

max n

207 of 245

208 of 245

209 of 245

210 of 245

211 of 245

212 of 245

213 of 245

1 billion agents

214 of 245

1 billion agents

215 of 245

agent-based model

beneficial

λ=1 in 100k

deleterious

λ=1 in 1k

poisson distribution

216 of 245

agent-based model

beneficial

p=1 in 100k

deleterious

λ=1 in 1k

n loci available

217 of 245

n loci

218 of 245

n loci

219 of 245

n loci

100 million agents

220 of 245

Generations

50/50

221 of 245

Generations

50/50

Generations

1 in 100k

denovo

222 of 245

1 in 100k

denovo

n loci

223 of 245

1 in 100k

denovo

n loci

224 of 245

1 in 100k

denovo

n loci

225 of 245

1 in 100k

denovo

n loci

226 of 245

1 in 100k

denovo

n loci

227 of 245

1 in 100k

denovo

n loci

228 of 245

1 in 100k

denovo

229 of 245

230 of 245

Next Steps

231 of 245

Next Steps

VS

osmotic pressure

antibiotics

Callens et al., 2023

232 of 245

Next Steps

VS

osmotic pressure

more adaptive loci

more hypermutators

antibiotics

fewer adaptive loci

fewer hypermutators

Callens et al., 2023

Q: how many beneficial muts per strain?

233 of 245

Next Steps

VS

osmotic pressure

more adaptive loci

more hypermutators

antibiotics

fewer adaptive loci

fewer hypermutators

Callens et al., 2023

Q: how many beneficial muts per strain?

more sophisticated fitness landscape models
well-mixed ABM on GPU
analyze time to fixation in model vs. theory expectation
create mathematical model for de novo selection curves
connect to data from Callens et al or LTEE?

234 of 245

Tagged, Event-driven Programming Model

N

E

S

W

“Ramp”

235 of 245

Tagged, Event-driven Programming Model

“Ramp”

N

E

S

W

“Wavelet”

236 of 245

Tagged, Event-driven Programming Model

“Ramp”

N

E

S

W

“Wavelet”

Queue

…

fn doTask(wav) {

int c = 10;

…

}

Task

237 of 245

[more about WSE programming model in pocket slides]

238 of 245

GraphCore IPU — another AI/ML accelerator

1,200 cores per chip
clustered up to 1,024 chips

📧 morenoma@umich.edu

239 of 245

Part 1: On-hardware Experiment

240 of 245

Scale and Digital Evolution

📧 morenoma@umich.edu

📚 [these slides] https://hopth.ru/ea

LTEE

(Good et al., 2017)

E.

coli

241 of 245

Scale and Digital Evolution

LTEE

(Good et al., 2017)

Avida

(Ofria and Wilke, 2009)

📧 morenoma@umich.edu

📚 [these slides] https://hopth.ru/ea

Image credit: Credit: National Institute of Allergy and Infectious Diseases, National Institutes of Health

E.

coli

242 of 245

Scale and Digital Evolution

both ~billion replications/day

LTEE

(Good et al., 2017)

Avida

(Ofria and Wilke, 2009)

📧 morenoma@umich.edu

📚 [these slides] https://hopth.ru/ea

Image credit: Credit: National Institute of Allergy and Infectious Diseases, National Institutes of Health

E.

coli

243 of 245

Scale and Digital Evolution

both ~billion replications/day

Cross-scale Phenomena:

ecological communities
multicellularity/major transitions

LTEE

(Good et al., 2017)

Avida

(Ofria and Wilke, 2009)

📧 morenoma@umich.edu

📚 [these slides] https://hopth.ru/ea

E.

coli

244 of 245

GPU

ALIEN Project (Heinemann, 2024)

MFM/ULAM/T2 Tiles (Ackley, 2023)

Illuninato x Machina

(Ackley, 2010)

📚 [slides, paper, code] https://hopth.ru/de

245 of 245

Objective: Develop Methods to Harness Next Generation AI/ML Hardware for Agent-based Evolution Experiments

📧 morenoma@umich.edu