The AGI Tech Stack

And building an AGI moonshot

Third Neural Scaling Laws Workshop 2022

​

Sonia Joseph

“AGI” is a supply chain interfacing with capitalism

Instead of being a single agent, or a single inflection point, “AGI” entwine will with our current incentive structures (e.g. Wall Street is already a superintelligence)

This is the slow take-off timeline.

Some key points worth questioning

• It’s more profitable to rely on an external API than building + training your own model
• It’s more efficient to do unsupervised pretraining before RL finetuning
• RL training on simulation does not transfer to robotics
• Hardware companies have more leverage than AGI startups, and the space may be less saturated
• Neuromorphic computing?

Where in the AGI stack to build a company?

How to start an AGI moonshot

• Understand founding stories of OpenAI, DeepMind, Vicarious, Numenta, & etc and examine larger patterns
• Master startup lore + organization building
• Flesh out AGI tech stack and go for counterfactual research bet
• Build a founding team with talent gene pool engineering
• Get the founding team building before taking capital
• Identify 2-3 research bets and build in those areas
• Slow fundraise process over ~2-3 years with aligned VCs and LPs
• Lay seeds
• Once you take capital, it’s a ticking clock
• Take capital from aligned parties
• Build AI alignment into organization design
• Alignment tax

Example: OpenAI

• Recruiting process was a series of lunches in the Bay Area
• Genetic diversity in founding team. Mix tech, business, and research acumen:
• Greg Brockman - previously CTO of Stripe (managed tech teams)
• Sam Altman - previously CEO of Y Combinator (the startup of startups)
• Ilya Sutskever - PhD University of Toronto w/ Geoff Hinton, DNNResearch, Brain (research)
• Elon Musk - aligned funding
• Organizational structure:
• Many teams, portfolio of bets toward AGI. Robotics, language, RL, & etc.
• Eventually collapsed into single technical direction: GPT-3 / LLMs

Example: DeepMind

• Startup <> research arbitrage
• Founding team had fundamental/maverick research + startup knowledge
• Demis Hassabis founded two video game companies before starting his third (DeepMind)
• Shane Legg - dissertation on Machine Superintelligence
• dissertation committee with Marcus Hutter, Jurgen Schmidhuber
• Mustafa Suleyman
• Like OpenAI, unclear what profit model would be in the beginning

Designing the organization

• Focused research organization (FRO)? For-profit startup?
• Look to biotech company models for drug development
• Drug development has 5-7 year timelines
• Look at deep tech best practices
• Profit models
• Eventually, FROs get acquired by tech supergiant
• Fund w/o acquisition
• Product/commercialization
• Government contracts

Thank you to Irina Rish, Blake Richards, Ethan Caballero, Gabriela Moisescu-Pareja, Brady Neal, Ankesh Anand, Taryn Livingstone, among others, for the conversations