The Course Book
Harvard CS197
AI Research Experiences
|
Pranav Rajpurkar PhD
Assistant Professor, Harvard University
Take your AI skills to the next level with this course.
Dive into cutting-edge development tools like PyTorch, Lightning, and Hugging Face, and streamline your workflow with VSCode, Git, and Conda. You'll learn how to harness the power of the cloud with AWS and Colab to train massive deep learning models with lightning-fast GPU acceleration. Plus, you'll master best practices for managing a large number of experiments with Weights and Biases. And that's just the beginning! This course will also teach you how to systematically read research papers, generate new ideas, and present them in slides or papers. You'll even learn valuable project management and team communication techniques used by top AI researchers. Don't miss out on this opportunity to level up your AI skills.
The following pages provide free online access to all 250 pages of course notes, as well as to course assignments and the final project.
Version 1.0, Jan 1, 2023
Copyright (c) 2023 Pranav Rajpurkar
Contents
You Complete My Sandwiches – Exciting Advances with AI Language Models 6
Harvard CS197 Lecture 1 Notes 6
The Zen of Python – Software Engineering Fundamentals 6
Harvard CS197 Lecture 2 Notes 6
Shoulders of Giants – Reading AI Research Papers 7
Harvard CS197 Lecture 3 Notes 7
In-Tune with Jazz Hands – Fine-tuning a Language Model using Hugging Face 7
Harvard CS197 Lecture 4 Notes 7
Lightning McTorch – Fine-tuning a Vision Transformer using Lightning 8
Harvard CS197 Lecture 5 Notes 8
Moonwalking with PyTorch – Solidifying PyTorch Fundamentals 8
Harvard CS197 Lecture 6 & 7 Notes 8
Experiment Organization Sparks Joy – Organizing Model Training with Weights & Biases and Hydra 9
Harvard CS197 Lecture 8 & 9 Notes 9
I Dreamed a Dream – A Framework for Generating Research Ideas 10
Harvard CS197 Lecture 10 & 11 Notes 10
Today Was a Fairytale – Structuring a Research Paper 10
Harvard CS197 Lecture 12 & 13 Notes 10
Deep Learning on Cloud Nine – AWS EC2 for Deep Learning: Setup, Optimization, and Hands-on Training with CheXzero 11
Harvard CS197 Lecture 14 & 15 Notes 11
Make your dreams come tuned – Fine-Tuning Your Stable Diffusion Model 11
Harvard CS197 Lecture 16 & 17 Notes 11
Research Productivity Power-Ups – Tips to Manage Your Time and Efforts 12
Harvard CS197 Lecture 18 Notes 12
The AI Ninja – Making Progress and Impact in AI Research 12
Harvard CS197 Lecture 19 Notes 12
Bejeweled – Tips for Creating High-Quality Slides 13
Harvard CS197 Lecture 20 Notes 13
Model Showdown – Statistical Testing to Compare Model Performances 13
Harvard CS197 Lecture 21 Notes 13
● Assignment 1: The Language of Code 14
● Assignment 2: First Dive in AI 14
● Assignment 5: Ideation and Organization 14
● Assignment 6: Stable Diffusion and Research Operations 14
Student Experiences
“CS197 is a must take if you're at all interested in anything ML/AI. Professor Rajpurkar does an amazing job of imparting years of wisdom gained through his experiences, and the course provides an invaluable background for anyone interested in understanding the field...This course has been one of the most interesting, applied courses that I've taken at Harvard, and while it definitely throws you into the deep end of AI, you come out having a tangible product and a solid set of skills.”
– Derek Zheng, Harvard Class of 2023
“If you are interested in applying Machine Learning theory to the real world, CS197 is a must take. In class you will not only learn about state-of-the-art models and how they work but also get familiar with industry-grade tools to help you become an active player in the field. Prior to the course ML jargon would usually throw me off and prevent me from jumping into new research papers. Now, however, I feel comfortable reading about all the new models coming out of OpenAI, DeepMind, Meta, Google, and academic institutions.“
– Ty Geri, Harvard Class of 2023
“The amount of content we covered over the semester was incredible, and I ended the semester knowing much more about practical tools in AI, research, and AI research. The course was incredibly hands-on, exposing us to the power we held in our hands even with an undergraduate level of expertise... This was a great course, and I would recommend others to take this course if you are interested in gaining more expertise in AI and/or want to explore research opportunities!”
– Sun-Jung Yum, Harvard Class of 2023
“I definitely found CS197 to be a challenging but extremely rewarding course. The class takes a learn-by-doing approach, which I really enjoyed and found unique compared to other classes. Overall, I would highly recommend this course to anyone new at research and/or deep learning and who would like to get more familiar with these topics!”
– Alyssa Huang, Harvard Class of 2024
“AI Research Experiences was truly the most thorough AI research course I've taken at Harvard. Exactly as advertised, Pranav walks us through his research philosophy, from software engineering principles like codebase setup to reading papers and forming our own hypotheses and running experiments... I would strongly recommend it to anyone looking for a thorough and guided introduction (or for others like me, reintroduction) to AI research - unlike any other course at Harvard, this course really forces one to become comfortable with a lot of modern AI research tools and practices.”
– Rajat Mittal, Harvard Class of 2023
Preface
I am truly excited and honored to share this course book with you. With over 250 pages of course notes, it is the culmination of my years of experience and expertise in the field, as well as my observations of what students new to AI often struggle with and the need to systematically train them in these tools.
In this course, you'll have the opportunity to learn from the best tools and technologies in AI, including PyTorch, Lightning, Hugging Face, VSCode, Git, and Conda. You'll also learn how to use the cloud with AWS and Colab to train massive deep learning models with lightning-fast GPU acceleration.
But this course is about more than just learning how to use the latest tools and technologies. It's about becoming a well-rounded AI researcher. You'll learn how to systematically read research papers, generate new ideas, and present them in slides or papers. You'll even pick up valuable project management and team communication techniques used by top AI researchers.
I am deeply grateful to the 16 students at Harvard who were selected to take the course in its first offering. Their insights and perspectives have been invaluable in shaping and improving the material. Additionally, I want to extend a special thank you to Elaine Liu, Xiaoli Yang, and Kat Tian for their contributions and support in the development and delivery of this course.
I hope that this course will inspire and guide you on your own journey in AI, and that it will serve as a valuable resource as you take your skills and career to the next level.
– Pranav Samir Rajpurkar PhD
Assistant Professor of Biomedical Informatics
Harvard Medical School
Chapters 1-2 (59 pages)
You Complete My Sandwiches – Exciting Advances with AI Language Models
Harvard CS197 Lecture 1 Notes
- Interact with language models to test their capabilities using zero-shot and few-shot learning.
- Learn to build simple apps with GPT-3’s text completion and use Codex’s code generation abilities.
- Learn how language models can have a pernicious tendency to reflect societal biases.
The Zen of Python – Software Engineering Fundamentals
Harvard CS197 Lecture 2 Notes
- Edit Python codebases effectively using the VSCode editor.
- Use Git and Conda comfortably in your coding workflow.
- Debug without print statements using breakpoints and logpoints
- Use linting to find errors and improve Python style.
Chapters 3-4 (41 pages)
Shoulders of Giants – Reading AI Research Papers
Harvard CS197 Lecture 3 Notes
- Conduct a literature search to identify papers relevant to a topic of interest
- Read a machine learning research paper and summarize its contributions
- Summarize previous works in an area
In-Tune with Jazz Hands – Fine-tuning a Language Model using Hugging Face
Harvard CS197 Lecture 4 Notes
- Load up and process a natural language processing dataset using the datasets library.
- Tokenize a text sequence, and understand the steps used in tokenization.
- Construct a dataset and training step for causal language modeling.
Chapters 5-7 (33 pages)
Lightning McTorch – Fine-tuning a Vision Transformer using Lightning
Harvard CS197 Lecture 5 Notes
- Interact with code to explore data loading and tokenization of images for Vision Transformers.
- Parse code for PyTorch architecture and modules for building a Vision Transformer.
- Get acquainted with an example training workflow with PyTorch Lightning.
Moonwalking with PyTorch – Solidifying PyTorch Fundamentals
Harvard CS197 Lecture 6 & 7 Notes
- Perform Tensor operations in PyTorch.
- Understand the backward and forward passes of a neural network in context of Autograd.
- Detect common issues in PyTorch training code
Chapters 8-9 (22 pages)
Experiment Organization Sparks Joy – Organizing Model Training with Weights & Biases and Hydra
Harvard CS197 Lecture 8 & 9 Notes
- Manage experiment logging and tracking through Weights & Biases.
- Perform hyperparameter search with Weights & Biases Sweeps.
- Manage complex configurations using Hydra.
Chapters 10-13 (23 pages)
I Dreamed a Dream – A Framework for Generating Research Ideas
Harvard CS197 Lecture 10 & 11 Notes
- Identify gaps in a research paper, including in the research question, experimental setup, and findings.
- Generate ideas to build on a research paper, thinking about the elements of the task of interest, evaluation strategy and the proposed method.
- Iterate on your ideas to improve their quality.
Today Was a Fairytale – Structuring a Research Paper
Harvard CS197 Lecture 12 & 13 Notes
- Deconstruct the elements of a research paper and their sequence.
- Make notes on the global structure and local structure of the research paper writing.
Chapters 14-17 (31 pages)
Deep Learning on Cloud Nine – AWS EC2 for Deep Learning: Setup, Optimization, and Hands-on Training with CheXzero
Harvard CS197 Lecture 14 & 15 Notes
- Understand how to set up and connect to an AWS EC2 instance for deep learning.
- Learn how to modify deep learning code for use with GPUs.
- Gain hands-on experience running the model training process using a real codebase.
Make your dreams come tuned – Fine-Tuning Your Stable Diffusion Model
Harvard CS197 Lecture 16 & 17 Notes
- Create and fine-tune Stable Diffusion models using a Dreambooth template notebook.
- Use AWS to accelerate the training of Stable Diffusion models with GPUs.
- Work with unfamiliar codebases and use new tools, including Dreambooth, Colab, Accelerate, and Gradio, without necessarily needing a deep understanding of them.
Chapters 18-19 (19 pages)
Research Productivity Power-Ups – Tips to Manage Your Time and Efforts
Harvard CS197 Lecture 18 Notes
- Learn how to use update meetings and working sessions to stay aligned and make progress on a project.
- Understand how to use various tools and techniques to improve team communication and project organization.
- Learn strategies for organizing your efforts on a project, considering the stage of the project and the various tasks involved.
The AI Ninja – Making Progress and Impact in AI Research
Harvard CS197 Lecture 19 Notes
- Learn how to make steady progress in research, including managing your relation with your advisor, and skills to develop.
- Gain a deeper understanding of how to increase the impact of your work.
Chapters 20-21 (25 pages)
Bejeweled – Tips for Creating High-Quality Slides
Harvard CS197 Lecture 20 Notes
- Apply key principles of the assertion-evidence approach for creating effective slides for talks.
- Identify common pitfalls in typical slide presentations and strategies for avoiding them.
- Apply the techniques learned in this lecture to real-world examples of research talk slides to improve their effectiveness.
Model Showdown – Statistical Testing to Compare Model Performances
Harvard CS197 Lecture 21 Notes
- Understand the different statistical tests that can be used to compare machine learning models, including McNemar's test, the paired t-test, and the bootstrap method.
- Be able to implement these statistical tests in Python to evaluate the performance of two models on the same test set.
- Be able to select an appropriate test for a given research question, including tests for statistical superiority, non-inferiority, and equivalence.
Assignments
These assignments allow you an opportunity to get hands-on experience applying the skills you have learned in the lectures. I encourage you to go through these assignments to solidify your understanding. To preserve the integrity of the assignments for future students, I encourage you to not make your solutions available publicly.
Assignment 1: The Language of Code
Assignment 2: First Dive in AI
Assignment 3: Torched
Assignment 4: Spark Joy
Assignment 5: Ideation and Organization
Assignment 6: Stable Diffusion and Research Operations
Course Project
For the course project, you will build on cutting-edge research applying artificial intelligence to medicine. We provide you with a clearly defined research direction and let you formulate a research problem and conduct end-to-end research. You utilize research tools and technical skills we learn in class to complete this project. From this final project, you will gain valuable experience of doing real research in medical AI, and tackle a project from conception to a full manuscript.
Project Details
Congratulations
Congratulations on completing this comprehensive course! You have worked hard to master the tools and techniques of AI research, and I am confident that you are now well-equipped to take on new challenges and opportunities in the field.
I encourage you to share your experience with this course on social media, using the hashtag #HarvardCS197. It would be great to see and hear about the progress you have made and the impact this course has had on your work and career.
Additionally, I invite you to use this form to provide feedback and suggestions for improving the course in the future. Your input is valuable and will help ensure that this course continues to be a valuable resource for aspiring AI researchers.
I want to take this opportunity to thank you once again for your dedication and hard work. I hope that this course has been an inspiring and rewarding experience for you, and I look forward to seeing the great things you will accomplish in the future.
Keep up the excellent work!