| A | B | C | E | F | G | H | I | J | ||
|---|---|---|---|---|---|---|---|---|---|---|
1 | Name with link | Summary | Structure - words | Length start to finish | Time per week | Social aspect built in? | Signing up | General notes | ||
2 | Add courses not listed or suggest updates | |||||||||
3 | AGI safety fundamentals program or here | 3.5 hours reading, 1.5 hours talking a week w/ facilitator for 8 weeks. Focuses on technical safety more than others | Meet once a week. Project at the end. | 8 weeks | 4.5 | yes | https://airtable.com/shr41jcUXUc62n1SM | Week 0 (optional): introduction to machine learning Week 1: Artificial general intelligence Week 2: Goals and misalignment Week 3: Threat models and types of solutions Week 4: Learning from humans Week 5: Decomposing tasks for outer alignment Week 6: Other paradigms for safety work Week 7: AGI safety in context Week 8 (several weeks later): Projects Further material | ||
4 | Alignment 201 Curriculum | This curriculum is intended as a follow-up to the Alignment Fundamentals curriculum (which is the ‘101’ to this 201 course). The course consists of 7 weeks of readings, then two weeks focused on developing a literature review and/or research proposal. As in the Alignment Fundamentals course, participants are divided into groups of 4-6 people, matched based on their prior knowledge about ML. Each week, each group and their discussion facilitator will meet for 1.5 hours to discuss the readings and exercises. Weeks 6 and 7 branch into three parallel tracks: participants can choose whether to focus on Eliciting Latent Knowledge, Agent Foundations or Science of Deep Learning. The main focus each week will be on the core readings and one exercise of your choice out of the exercises listed, for which you should allocate around 2 hours preparation time. Approximate times taken to read each piece in depth are listed next to them. | Meet once a week. Project at the end. | 9 weeks | 4 | yes | https://www.agisafetyfundamentals.com/participate-details | |||
5 | AI safety camp | Actually do some AI research. More about output than learning. | preparation (≧ 7 h/week for 7 weeks), weekend sprints (≧ 8 h/d), project (5-15 h) | 7-8 weeks | ≧ 7 h during preparation | yes | https://aisafety.camp/pre-sign-up/ | |||
6 | Introduction to ML Safety | A course covering topics in machine learning relevant to ML safety. | It includes recorded lectures, written assignments, coding assignments, and readings. In ML Safety Scholars it was used as part of a structured program but otherwise it's an online course. | 40 hours | n/a | no | n/a | |||
7 | How to pursue a career in technical AI alignment | A reading list that includes both reading recommendations as well as general advice about how to get a career in the field such as whether you need a PhD, etc. | Reading list | n/a | n/a | no | n/a | |||
8 | Victoria Krakovna resources | Big collection of varied resources, starting with basic overviews and career advice and then jumps into technical research papers | n/a | 40 weeks | n/a | no | n/a | |||
9 | How to get into independent research on alignment | Guide on how to do independent research specifically. | Blog post | 10 minutes | n/a | no | n/a | |||
10 | List of longtermist courses | List of courses related to longtermism more generally, not AI safety specifically | n/a | n/a | n/a | n/a | n/a | |||
11 | AI Alignment Forum Sequences | Written by Ngo, Shah, and Christiano. They probably have a lot of overlap with Ngo's fellowship | Similar to the rationality sequences | 8 weeks | n/a | no | n/a | |||
12 | CHAI recommended reading materials | List of resources prioritized by importance to read. | n/a | 40 weeks | n/a | no | n/a | |||
13 | 80k reading syllabus | Lots of math, game theory, and textbooks about fundamental theory | n/a | 40 weeks | n/a | no | n/a | |||
14 | Reading group | No structure reading group. Has really good YouTube channel where they record the summaries of the papers they read. | Reading group. No curriculum | inf | 1 | yes | The AISafety.com Reading Group meets weekly, usually Thursdays at 19:45 UTC. To join, add “soeren.elverlin” on Skype. Usually, we start with small-talk and a presentation round, then the host gives a summary of the paper for roughly 20 minutes. The summary of the article is uploaded on the Youtube Channel. This is followed by discussion (both on the article and in general) and finally we decide on a paper to read the following week. | |||
15 | Safety and Control for Artificial General Intelligence | An actual AI Safety university course (UC Berkeley). Touches multiple domains including cognitive science, utility theory, cybersecurity, human-machine interaction, and political science. | Twice per week. Coding & final projects | 1 semester | 4 | yes | n/a | |||
16 | AI Safety Support "Lots of Links" | A huge collection of resources of many categories, some of them have no overlap with other collections (e.g. Funding, Landscapes, Talks and more) | n/a | 40 weeks | n/a | no | n/a | |||
17 | Awesome Artificial Intelligence Alignment | List of resources organized by type of content. Good source of events, podcasts, and general resources | n/a | 40 weeks | n/a | no | n/a | |||
18 | Bibliography of all safety research | Not a course, but a comprehensive list of all papers put out in safety. | Bibliography | 60 weeks | n/a | no | n/a | |||
19 | AI alignment newsletter | Not a course, but a great resource filled with summaries and commentary on safety papers | Newsletter | inf | n/a | no | newsletter signup | |||
20 | Vael's recommended reading list | A more general list including why to work on AI safety in the first place and also more in depth technical recommendations, governance, biosecurity, etc. | Blog post | 10 weeks | n/a | no | n/a | |||
21 | Neel Nanda's Inside View Resources | Not a course, but a great resource filled with ways to form an inside view in a new field. | List of resources | n/a | n/a | no | n/a | |||
22 | Existential Risks Introductory Course (ERIC) | Long distance course on general x-risks, including AI. Not a technical course. 1.5 hours a week meeting with fellow students and facilitators. | Long distance course | 8 weeks | ~2 hr reading 1.5 hr discussion | yes | Expression of interest form | |||
23 |