ABFGHIJKLMNOPQRSTUVWXYZAAAB
1
2
DateRecommended ResourcesAction itemsTODOPresentationsTAs
3
Prereqs (<6hr/day):
4
Wed: Env set-up + Linear algebra prereqsJuly 3rdARENA Reference: https://arena3-chapter0-fundamentals.streamlit.app/[0.0]_Prerequisites#linear-algebra

Note: We have not budgeted time for the other math subjects. Consider substituting probability/stats, calculus, or information theory for other prereq material with which you are more comfortable

General
-If you are rusty on linear algebra, a reasonable refresher is available here: https://www.youtube.com/watch?v=fNk_zzaMoSs&list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab.
-You will not have time to take a more fulsome approach, but a more traditional approach (to the extent you wanted to drill down more into a specific topic) is availble here: https://ocw.mit.edu/courses/18-06-linear-algebra-spring-2010/

Specifics:
-https://www.lesswrong.com/posts/iupCxk3ddiJBAJkts/six-and-a-half-intuitions-for-svd
- https://web.stanford.edu/~mrosenfe/soc_meth_proj3/matrix_OLS_NYU_notes.pdf

For Environment Set-up: https://gist.github.com/zachlipp/9ed6bac71a507c87acd716674a22c580#file-tech_setup-md
Read through the this link: https://github.com/changlinli/intro-to-technical-ai-safety-slides/blob/master/basic_linear_algebra_exercises/exercises.md and do the exercises on the page		Get a minimum example of set-up for people to run on the first
Week 1:



Day 3 (Feb 28th): Implement PyTorch backprop and tensors, modules, parameters, and more from scratch: https://arena3-chapter0-fundamentals.streamlit.app/[0.4]_Backprop
Day 4 (Feb 29th): Build transformers from scratch (a.k.a. build your own GPT): https://arena3-chapter1-transformer-interp.streamlit.app/[1.1]_Transformer_from_Scratch
Day 5 (Mar 1st): Intro to mechanistic interpretability (a.k.a. understanding why models say the things they do): https://arena3-chapter1-transformer-interp.streamlit.app/[1.2]_Intro_to_Mech_Interp

Week 2:

Day 6 (Mar 4th): More mechanistic interpretation. Have some options here. Can decide to look at how a transformer model is able to balance parentheses, how to use specially crafted vectors to "steer" a model's behavior, or how a model is about to identify the indirect object of a sentence (https://arena3-chapter1-transformer-interp.streamlit.app/[1.7]_Balanced_Bracket_Classifier or https://arena3-chapter1-transformer-interp.streamlit.app/~/+/[1.5]_Function_Vectors_&_Model_Steering or https://arena3-chapter1-transformer-interp.streamlit.app/[1.3]_Indirect_Object_Identification). These are long so you'll likely just end up doing part 1 of one of these especially if you choose indirect object identification.
Day 7 (Mar 5th): Finish one of the tasks from yesterday if you haven't yet, or do another.
Day 8 (Mar 6th): Tabular reinforcement learning and Deep Q-Learning culminating in teaching an agent how to keep a wobbly pole standing: https://arena3-chapter2-rl.streamlit.app/[2.1]_Intro_to_RL and https://arena3-chapter2-rl.streamlit.app/[2.2]_Q-Learning_and_DQN (both chapters are short so we'll combine them together)
Day 9 (Mar 7th): Using PPO + ResNets to train an agent to play the Atari game Breakout: https://arena3-chapter2-rl.streamlit.app/[2.3]_PPO
Day 10 (Mar 8th): Using RLHF to tune a language model's behavior: https://arena3-chapter2-rl.streamlit.app/[2.4]_RLHF
5
Friday: Tensor manipulationJuly 5th ARENA Reference: https://arena3-chapter0-fundamentals.streamlit.app/[0.0]_Prerequisites#libraries

General:
-https://pytorch.org/tutorials/beginner/basics/intro.html
-https://pytorch.org/tutorials/beginner/nn_tutorial.html

Specifics:
-https://medium.com/analytics-vidhya/pytorch-contiguous-vs-non-contiguous-tensor-view-understanding-view-reshape-73e10cdfa0dd
-skim: https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf		https://github.com/peluche/pre-arena

-At a high level, what is a torch.Tensor?
-What is a nn.Parameter, and nn.Module?
-When you call .backward(), where are your gradients stored?
-What is a loss function? In general, what does it take for arguments, and what does it return?
-What does an optimization algorithm do?
-What is a hyperparameter, and how does it differ from a regular parameter?
-What are some examples of hyperparameters?

https://github.com/rougier/numpy-100/blob/master/100_Numpy_exercises.ipynb (optional)
Regis Schiavi
6
Prereqs cont'd + Fundamentals
7
 Monday: Einops p1, p2July 8thARENA Reference:

General:
-https://einops.rocks/1-einops-basics/
-https://einops.rocks/api/einsum/
-https://rockt.github.io/2018/04/30/einsum

Specifics:
-https://arena3-chapter0-fundamentals.streamlit.app/~/+/[0.0]_Prerequisites#einops-einsum-tensor-manipulation		1. https://colab.research.google.com/drive/1TCd8ujgWqD9LOBG4WBCcW8mklvK4fWBZ?usp=sharing#scrollTo=5-mRNVeE9cDh (scroll down past answers at the top to view the questions)
2. Work through einops exercises on this page. Click einops in the bottom left corner: https://arena3-chapter0-fundamentals.streamlit.app/[0.0]_Prerequisites
8
Tuesday: Neural networks from scratchJuly 9thhttps://karpathy.ai/zero-to-hero.html (especially through part 4; it isn't practical to watch this all in a day and even those 4 parts is too much)https://github.com/changlinli/neural-nets-from-scratch
9
Wednesday: 0.2 CNNs and ResNetsJuly 10thhttps://arena3-chapter0-fundamentals.streamlit.app/[0.2]_CNNs_&_ResNets
https://colah.github.io/posts/2014-07-Conv-Nets-Modular/
Create a ResNet model to do more general image classification (and implement some common PyTorch modules. On Day 3 we'll implement the actual Module class from scratch): https://arena3-chapter0-fundamentals.streamlit.app/[0.2]_CNNs_&_ResNets
https://colab.research.google.com/drive/1sZpsnjN7gI4EucRcD0mCafN5Vg1DPOnm
Thien
10
Thursday: 0.2 CNNs and ResNetsJuly 11th
11
Friday: 0.3 - OptimizationJuly 12th
Create PyTorch's optimizers from scratch and do hyperparameter sweeps with Weights and Biases: https://arena3-chapter0-fundamentals.streamlit.app/[0.3]_Optimization
12
Monday: 0.3 - OptimizationJuly 15th
13
Tuesday - Pause for Impossible day!July 16th
14
Wednesday: 0.4 BackpropJuly 17th
Implement PyTorch backprop and tensors, modules, parameters, and more from scratch: https://arena3-chapter0-fundamentals.streamlit.app/[0.4]_Backprop
15
Transformers and Mech Interp
16
Thursday: 1.1 - Transformers from ScratchJuly 18thhttps://karpathy.ai/zero-to-hero.html: https://www.youtube.com/watch?v=kCc8FmEb1nY, https://youtu.be/zduSFxRajkE
Build transformers from scratch (a.k.a. build your own GPT): https://arena3-chapter1-transformer-interp.streamlit.app/[1.1]_Transformer_from_Scratch
17
Friday 1.1 - Transformers from ScratchJuly 19thhttps://karpathy.ai/zero-to-hero.html: https://www.youtube.com/watch?v=kCc8FmEb1nY, https://youtu.be/zduSFxRajkE
Build transformers from scratch (a.k.a. build your own GPT): https://arena3-chapter1-transformer-interp.streamlit.app/[1.1]_Transformer_from_Scratch
18
Monday: 1.2 - Intro to Mech Interp July 22
Intro to mechanistic interpretability (a.k.a. understanding why models say the things they do): https://arena3-chapter1-transformer-interp.streamlit.app/[1.2]_Intro_to_Mech_Interp
Feel free to continue with Transformers intro if you haven't finished
19
Tuesday 1.2 - Intro to Mech InterpJuly 23
20
Wednesday: 1.3 - Indirect Object Identification
OR 1.5 - Function Vectors and Model Steering
OR 1.7 - Balanced Bracket Classifier		July 24
21
Thursday: 1.3 - Indirect Object Identification
OR 1.5 - Function Vectors and Model Steering
OR 1.7 - Balanced Bracket Classifier		July 25
22
Friday 1.3 - Indirect Object Identification
OR 1.5 - Function Vectors and Model Steering
OR 1.7 - Balanced Bracket Classifier		July 26
23
Week 4 - Reinforcement Learning
24
Monday: 2.1 - Intro to Reinforcement Learning July 29
25
Tuesday 2.2 Q-Learning and DQNJuly 30
Tabular reinforcement learning and Deep Q-Learning culminating in teaching an agent how to keep a wobbly pole standing: https://arena3-chapter2-rl.streamlit.app/[2.1]_Intro_to_RL and https://arena3-chapter2-rl.streamlit.app/[2.2]_Q-Learning_and_DQN (both chapters are short so we'll combine them together
26
Wednesday 2.3 - PPO July 31
27
Thursday : 2.4 - RLHFAugust 1
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100