"Commercial Exhaust" Policy

Why?

• Frontier labs and state actors are locked in a race to develop systems more intelligent than humans, and are plausibly on track to achieve this during the coming decade.

• It is at best uncertain whether or not techniques for enabling people to wield superintelligence in support of their goals will be developed until then.
• Even if such techniques are developed, the first player to develop them might get a decisive grip on society more broadly.

• What success ought to mean for us, then, is making it more likely, even if marginally, that the development of such systems goes well for the world.

• At any one point, this would naturally translate to work on concrete interventions, such as capability demonstrations or coordination technology, each motivated by a particular theory of change, such as communicating risks or encouraging cooperation.
• However, regardless of how developments impact our priorities, marginally contributing to making AI go well would in itself constitute a huge success.

• If we are to also achieve some degree of commercial success as an R&D lab, that is merely a nice-to-have bonus.

• It might be a significant achievement in its own right, but it pales in comparison to the core mission.
• I'd much rather we succeed at making it a tiny bit more likely that we'll be around in a decade, than enjoy some short-lived hypergrowth.

What?

• The way we're integrating the two objectives is by reducing commercial efforts to a matter of opportunistically exploring near-term applications of technology developed in support of advancing the core mission.

• Our procedural cyber range primarily helps with capability demonstrations, yet can secondarily enable more mundane applications such as security training.
• Our capability access controls primarily help with facilitating cooperation, yet can secondarily help make enterprise agent deployments more secure.
• Put differently, the R&D process which is fundamentally aiming to move the needle on catastrophic risks can also generate "commercial exhaust."

How?

It'd be wise to note that similarly-sounding plans have historically fared quite poorly. Talk is cheap, and "we'll only use product to support safety research" is getting old pretty quickly. So, what's different this time around?

1. I'm investigating the technicalities of a Founder's Pledge, a legally-binding commitment for donating a proportion of proceeds associated with the entity to charity (e.g., during an exit). Committing to one configured at 100% would essentially make a supermajority of ownership financially disinterested, while preserving the operational flexibility of a for-profit.
2. Legally-binding clauses for internally incentivizing the upholding of financial independence from organizations contributing to the race dynamics, such as by conditioning vesting on an operationalization of financial independence at a given point in time, or triggering the very dissolution of the entity when that criterion is not upheld.

99% of startups fail, so thinking about having to wrestle with this impact-growth tension might feel self-indulgent. However, it is in 100% of the cases that actually matter (i.e., in that outsized 1%) that this thinking is actually essential. What does a reputational hit of coming across as self-aggrandizing even signify relative to being better prepared in the situations that actually matter?