Technical AI Forecasting Resolution Council -- expression of interest
This form is an expression of interest for membership of the first round of the AI Resolution council.
Note that this round will last for ~1 year and likely be more experimental than future rounds, as we try to figure out what structures work best for this type of project.
BACKGROUND AND MOTIVATION
Getting well-calibrated forecasts about the future of AI is important to both policymakers, grantmakers and researchers in AI safety and policy.
Forecasting platforms such as
have proven successful for this purpose, but they face an important bottleneck: questions must be clearly resolvable. This is hard for several important questions:
1. *Counterfactual questions* E.g. consider a question regarding an AI system successfully passing a specific capability test. The test might be made very precise, but there is no guarantee the exact test described would actually be performed, so resolution would be ambiguous even if it were quite clear that a system that could pass the test exists.
2. *Similarity questions* E.g. If one benchmark becomes replaced by another, can we compare performance of algorithms across the two? If RL agents are trained on a new version of Dota, can we compare performance to OpenAI Five’s on Dota 2?
3. *Definition-of-terms questions* E.g. what counts as “hard-coded domain knowledge”? How much reward shaping can you add before the system no longer learns from “first principles”?
4. *Valuation questions* E.g. “How plausible does research agenda X seem?”, “How impressive was progress in field Y?”
Composing an expert council of senior AI researchers could serve as an “oracle” for these questions. The introduction of such a resolution mechanism would provide a rich expansion to the set of forecastable questions.
Moreover, the council would be an independent entity, providing open-source evaluations as a public good to be used by any forecasting project. (Current organisations with AI forecasting projects including Metaculus, the Open Philanthropy Project, OpenAI, and more.)
STRUCTURE OF COUNCIL
*Composition:* Initially >=5 members, in future possibly >=20. Members should:
---Have a technical background in AI (e.g. at least be PhD students)
---Represent a range of organisations
---Represent a range of technical fields (e.g. RL, unsupervised learning, NLP, safety, …)
---Have some understanding of forecasting (in particular, the importance of operationalizations and the possibility of divergence between “the letter and spirit of the law”)
*Frequency of meetings:* once per question batch, which might occur roughly every quarter
*Estimated time-commitment per member & question batch:* 1 hour for initially evaluating questions, 2 hours for settling on the resolution of all questions that need resolving, and time-value of overhead cost ~2-6 hours
*Compensation:* members will be payed a symbolic honorarium, amount TBD
*Responsibility of members:* sign a public pledge to commit to answering questions to the best of their ability at a pre-specified date; then to attend the meeting and vote on the resolution (there are no requirements for justification)
*Support and management:* operations for the council will be handled by Parallel, LLC (Jacob Lagerros and Ben Goldhaber), supported by a grant from the Berkeley Existential Risk Initiative (
). Parallel are also responsible for preparing pre-reading material for the meetings to inform decisions and save time; as well as filtering which requests are included in the evaluation batch
*Transparency:* at least membership of the council is public knowledge. Proceedings might follow Chatham House rules
*Opinion aggregation method:* For ordinary binary questions, majority vote (the only fair social choice function if we require a binary resolution)
*Deliverable:* Council verdicts are added to the AI Resolution dictionary to inform future cases, and this dictionary is made open-source available to any forecasting project
Please address any questions to:
What is your experience in AI?
Feel free to be brief or to just link to a LinkedIn profile/other online resume. Please include: *educational background, *work experience, *publications (or high quality blog posts), *areas of expertise
What is your experience with forecasting?
At the time of the first resolution, council members must have some understanding of forecasting -- in particular, the importance of operationalizations and the possibility of divergence between “the letter and spirit of the law”. We will offer an initial training session in this area (and continual support) for members with little prior knowledge.
I understand that by joining the council I will have to recuse myself from any forecasting activities involving the output of the council.
Note: you will *not* automatically join the council by submitting this form, which is just an expression of interest.
I understand that by joining the council I will sign a public precommitment to judging cases at a pre-specified date, in order for forecasters to trust the stability of the council.
*Note:* you will *not* automatically join the council by submitting this form, which is just an expression of interest.
Anything else you'd like to point out?
If you'd like to recommend someone else as a plausible member of the council, add their names and emails here.
Send me a copy of my responses.
Never submit passwords through Google Forms.
This content is neither created nor endorsed by Google.
Terms of Service