Announcements: Podcast Assignment

Podcast assignment (new to CS195 since I last taught it).

• “You will record a 7-10 minute audio file of you talking with one or two other people in the course, then peer review two other podcasts.”

​

​

Podcast Assignment group matching form coming out next week.

Announcements: Clearview AI

During ~week 5, we’ll cover Privacy.

• Please request Clearview AI’s file on you: https://privacyportal.onetrust.com/webform/1fdd17ee-bd10-4813-a254-de7d5c09360a/2a09e1a7-f09f-4e0c-91a2-5818abe414d5
• One of the required “readings” will be their file on you.
• It’s unclear if they will accept requests from foreign nationals who are temporarily residing in California. But let’s try!�

Computing in the News (Wired)

4

Petition (that got emailed to me last night by the DailyCal)

I was asked to comment on this petition. Maybe 195 should discuss?

DOGE: Future Discussion

Tentatively: I’m hoping to talk about DOGE in more detail next week!

Note (added Feb 8, 2025): It’ll be a while. I’m holding a smaller discussion first next week, so I can figure out how to hold a productive discussion given how spicy the topic is.

​

​

Also, interesting story: Someone thought I was asking students to sign the petition since I included a screenshot of the petition. Nope. I ended up having a supervisor reach out to me, talk to me twice, then get in writing that I had never heard of the petition before the DailyCal emailed it to me for comment, and to also get in writing that am not the person who created the petition (???), and to also get in writing that I did not ask students to sign the petition during class. Spicy topic indeed!

Education

Feburary 3, 2025

CS 195, Spring 2025 @ UC Berkeley

Josh Hug https://cs195.org/sp25

7

WEEK 03

Open Question (before we get started)

Why are you in college instead of just learning from online resources?

• Community and connections you build.
• Working in teams.
• Direct access to professors. Interactivity, bidirectionality. You don’t know what you don’t know, professors help guide you.
• Proof of learning, credential!
• Social cues / socializing.
• Accountability. There are grades, structure, deadlines.
• I liked some lecturers a lot.
• ​

Impact (question via Brian Harvey)

Which of the following technologies have had the biggest impact on education?

​

1. Web search
2. Large language models
3. Word processing
4. Polling technologies (iClicker, Zoom Polls, etc.)
5. Social networking

​

https://www.yellkey.com/treat

​

Peer Instruction (thanks to BH)

“Multiple choice tests have changed what counts as knowledge in schools. Open-ended questions were the norm 30 years ago. The kind of knowledge you can report on multiple-choice tests is unimportant in the big scheme of things, and what’s really important is not what you already know, but how you can take what you already know and apply it something you’ve never seen before. Multiple choice tests make that hard. Teaching follows tests! The folks who invented Standardized Testing didn’t foresee how it would affect what knowledge means! (unintended consequence)”�– Brian Harvey

Impact (question via Brian Harvey)

There are 100 prisoners in solitary cells. There's a central living room with one light bulb; this bulb is initially off. No prisoner can see the light bulb from his or her own cell. Everyday, the warden picks a prisoner equally at random, and that prisoner visits the living room. While there, the prisoner can toggle the bulb if he or she wishes. Also, the prisoner has the option of asserting that all 100 prisoners have been to the living room by now. If this is indeed true, all prisoners are set free, but if it is false, they lose the chance to ever be freed. Thus, the assertion should only be made if the prisoner is 100% certain of its validity. The prisoners are allowed to get together one night in the courtyard, to discuss a plan. What plan should they agree on, so that eventually, someone will make a correct assertion?

​

...Now imagine if I gave you four algorithms and you were asked to find the correct one. Checking is different than generating!

​

Hype

12

Lecture 03, CS 195, Spring 2025

The Promise of Film Based Education

“I believe that the motion picture is destined to revolutionize our educational system, and that in a few years it will supplant largely, if not entirely, the use of textbooks. I should say that on the average we get only about two percent efficiency out of textbooks as they are written today.

​

The education of the future, as I see it, will be conducted through the medium of the motion picture, a visualized education, where it should be possible to obtain one hundred percent efficiency.” - Thomas Edison, 1922

The Promise of Radio Based Education

“The time may come when a portable radio receiver will be as common in the classroom as is the blackboard. Radio instruction will be integrated into school life as an accepted educational medium.” - William Levonson, directory of Cleveland public school’s radio station, 1945.

​

The Promise of MOOCs

• “Take the best courses from the best instructors at the best universities, and provide it to everyone around the world for free.” – Daphne Koller (cofounder of Coursera) (2012)

​

• “In the early years of the third millennium, most professors are still teaching in virtually the same way they were taught and their teachers were taught, stretching back centuries. Technology is transforming (if not threatening to overwhelm) higher education. We are on the verge of a transformation on the scale of that wrought by Gutenberg.” – Bob Sedgewick, Princeton University (~2012)

​

From “The Flickering Mind” by Todd Oppenheimer (Link)

“In a 1986 book, Teachers and Machines: The Classroom Use of Technology Since 1920, Larry Cuban, a professor of education at Stanford University and a former school superintendent, observed a pattern in how schools handled each round of technology.”

• “The cycle always began with big promises, backed by the technology developers' research. In the classroom, teachers never really embraced the new tools, and no significant academic improvement occurred.”
• “This provoked consistent responses from technology promoters: The problem was money, or teacher resistance, or the paralyzing school bureaucracy.“
• “Meanwhile, few people questioned the technology advocates' claims. As results continued to lag, the blame was finally laid on the machines.“

Your Experience

Have you ever been forced to use a piece of educational technology that seemed to fall short of its promise?

• IXL: Gives same basic help for every question. Supposed to help teach you, but it confuses you. Has smart scoring system, if you get 10 in a row right, but if you get one wrong, you get put back in the cycle (up to maybe 30).
• Very popular.
• But some felt it was “clean and simple”, “gamified, rewards for completing lessons”, “same thing again”
• But: Saw it in my nightmares.

​

Your Experience

Have you ever been forced to use a piece of educational technology that seemed to fall short of its promise?

• WebAssign: textbooks problems.
• Albert.io
• Scratch: Promoted “bad” coding practices. Workflow felt slow.
• Snap!: Liked it as a conceptual introduction to logic, control.
• CS61B autograder is very good.
• Ultimate Review Packet: For some courses it was good, some were bad.

​

​

​

Your Experience

Did anybody try online programming education before they took any formal classes? What did you try? How did it go?

• Youtube: Java desktop applications, watched tutorials. Get stuck in a loop with tutorial where you don’t get to really explore. Own projects are better.
• Scratch messing around on my own: Built a few small things for competitions, got into First Lego League (FLLs). Lots of youtube.
• 2013 Khan Academy - p5.js, you get a canvas, can draw things. Could immediately be creative without having to go through setup.
• Khan academy: Their javascript tutorial was pretty good. Could access other people’s projects, ended up playing games in other people’s projects.

On the Other Hand...

There are plentiful online resources out there!

• In CS: FreeCodeCamp, code.org, leetcode, 61A website, 61B website, etc.
• In education more generally: Khan Academy, 3 Blue 1 Brown, Coursera, EdX

Post-Hoc Analysis of Dan Garcia’s BJCx (Online CS10)

By and large, most of the students were:

• From either the United States (35%) or India (9%)
• Had at least a bachelor’s degree.

I saw similar results with the Princeton algorithms MOOC.

Udacity

Early experiments with MOOCs for the masses ended poorly.

• 74% in traditional classes passed, < 51% of Udacity students passed any of the three courses @ San Jose State.

Udacity has embraced a model focused on advanced education.

• Much easier to make money providing training for corporate employees.
• Easier to do a good job.
• Example: Self-driving cars

The Rise of Online Education

Strapped for cash, universities have followed suit offering for-profit online education programs.

But there are examples where students seem perfectly happy even at high price points:

• https://medium.com/towards-data-science/is-uc-berkeley-mids-worth-it-5ac645aa79f8

Any Thoughts?

Expensive professional master’s programs: Necessary evil for universities. Get money from some students to pay for other things (including “traditional students”).

​

Difficult to judge the value that you get out of these programs / courses.

�UC Berkeley Cyber Security masters: Started filling out application, called me in the morning, we saw you started filling it out. Dollars per unit figure. Did math, that’s $80k. Paying a bit for the name, name might not be that much different.

​

As an undergrad: More EE. Most of my degree value here was access to cutting edge resources, microfab classes, circuit classes. Industry experience, immediately carried over. Way more experience here than elsewhere. Companies value online masters, but undergrad from here is better.

​

Any Thoughts?

Online degrees are great for people later in life: You can do classes while also raising a family without having to commit to being somewhere physical.

1-on-1 Instruction and Mastery Learning

26

Lecture 03, CS 195, Spring 2025

Exam Curves

Below, we see everyone’s friend, the exam curve.

What does it mean to be here?

Or here?

Or here?

Or here?

Education

The most common model of education is a fixed-time, variable-learning model, where the entire class moves synchronously through a curriculum.

• Assessment is done on a fixed, one-shot schedule.
• If you fall behind, too bad.

​

This is done for cost reasons.

• Teacher to student ratios are too large to allow desynchronization.

​

The obvious downside is that some students fall behind, while others are learning less than they should be.

The Zone of Proximal Development

One well known idea in education is the Zone of Proximal Development.

​

As educators, we try to keep you in that middle region.

• But if everyone’s current abilities vary, this is hard.

​

Teaching Observation from CS61B (old data, still holds true)

Age people started programming is a strong predictor for 61B GPA...

​

Doing Well in College

Two extreme views:

• Nature: Raw talent is by far the dominant factor in college performance.
• Nurture: With proper education, anyone could perform at the very top level in college courses.

​

Learners have a certain probability distribution based on:

• Ability to properly study, or as Sahai calls it “Will to Prepare to Win”.
• Prior exposure to similar material / abstractions.
• Raw talent, sometimes called ‘fluid intelligence’.
• Ability to tap into the human network.
• And more…

Could everyone do well? How well?

Growth vs. Fixed Mindset

Students can be thought of as having either a “growth” mindset or a “fixed” mindset (based on research by Carol Dweck).

• “In a fixed mindset students believe their basic abilities, their intelligence, their talents, are just fixed traits. They have a certain amount and that's that, and then their goal becomes to look smart all the time and never look dumb.” - Distribution is narrow / has a low cap.
• “In a growth mindset students understand that their talents and abilities can be developed through effort, good teaching and persistence. They don't necessarily think everyone's the same or anyone can be Einstein, but they believe everyone can get smarter if they work at it.” - Distribution is wide / has a high cap.

​

Bloom’s Two Sigma Problem (Bloom 1984)

Core observation: An “average” student learning in a “Tutorial” format (1-on-1 with mastery) achieves results similar to the top 2% of a lecture based class (two standard deviations above the mean).

Original paper: “The Search for Methods of Group Instruction as Effective as. One-to-One Tutoring”

• Later studies have replicated results to various degrees.
• Tutorial-based education is far too expensive.

Bloom’s Two Sigma Problem (Bloom 1984)

Core observation: An “average” student learning in a “Tutorial” format (1-on-1 with mastery) achieves results similar to the top 2% of a lecture based class (two standard deviations above the mean).

A “mastery learning” course requires students to pass a checkpoint before proceeding.

• The key idea (IMO) is immediate feedback and corrective action.
• Can be much more logistically complex (e.g. imagine 61A entirely taught this way).

Bloom’s Two Sigma Problem

Bloom’s Two Sigma Problem: In one experiment, student randomly picked for 1-on-1 teaching performed similarly to the top 2% of a simultaneous lecture course.

2σ

7661.43

Top 2%

7216.70

61A/88 Grade vs. 61B Grade

From “Analysis of Factors and Interventions Relating to Student Performance in CS1 and CS2” by my former grad student:

A Supporting Experiment for “Everyone Can Do Well”

In Sp16, I gave students the option to fail intentionally.

• On average, students were 2.5 letter grades higher the second time, e.g. someone in the middle of the B range got an A the second time.

Bloom’s Two Sigma Problem (Bloom 1984)

Technological tools may help us:

• Get closer to Bloom’s Two Sigma goal through some form of automated instruction.
• Dramatically broaden access to education.

​

​

​

​

​

… IMO turn-key online education solutions seem doomed.

• But give amazing opportunities for adding feedback.
• Ex: Getting homework problems wrong? Go watch a Khan Academy video or ask an LLM (more on this later).

Open Question

Has anyone taken a class of Dan Garcia’s that has mastery learning as a guiding principle “(Achieving "A's for All (as Time and Interest Allow)")”? How did it go?

• I liked the 61C class.
• Really liked unlimited extensions.
• ​

Breakout Discussion

Where do you lie on the question of nature vs. nurture in determining academic success? If nurture, how early?

• (this is in part a warmup to next week)

🤔🤔🤔

Next Up: LLMs

Let’s take a break and fill out the attendance form.

​

Attendance: Head to bcourses.berkeley.edu and click the assignment called “Week 3 (Education): Attendance”.

• Or www.yellkey.com/young
• Enter the code and submit.
• In future weeks we may have more than one attendance code.

​

You must submit by the end of lecture (12:30) plus a short grace period.

41

Lecture 03, CS 195, Spring 2025

LLM

How much have you used LLMs to support your education?

• In 70, had classes during OH. Asked Chat to understand.
• Realized I was missing knowledge, asked ChatGPT to make a curriculum for me.

​

How much have you used LLMs to avoid doing tasks that you don’t want to do?

• Write emails for clubs.

​

Khanmigo (Video Clip 1)

​

Khanmigo (Video Clip 2)

​

Breakout Discussion (if the vibes are right)

Question 1: Do you think LLMs are already capable personal tutors? Do you think they will be eventually? When?

​

Question 2: Will LLMs significantly close the education gap? (In contrast to say, MOOCs, which we saw earlier tend to only reach the already-educated)

​

Question 3: Would you personally enjoy working on LLMs for education? Why or why not?

Education Platforms (Time Permitting)

46

Lecture 03, CS 195, Spring 2025

Platforms

There is an absolutely gigantic market for education.

• Thousands of technology companies have tried their hand at producing useful tools.
• Tools vary from small scale (automated homework assessment) to huge (all-in-one platforms with prepackaged curricula).

​

Coronavirus has dramatically expanded the need for such tools, at least in the short term.

​

Let’s discuss a few hits and misses.

• Keep this in mind if you ever go into educational software!

​

Example: K12

From our Wired reading, the 345,000 student Miami-Dade school district adopted the platform K12 (https://www.k12.com/) for mandatory use by all students.

• In response to Coronavirus, they needed tools to go online, and fast.
• Had anticipated resuming in person schooling, but cases spiked.
• K12 was supposed to be an all-on-one platform that did everything (even video conferencing).

​

​

​

(Note: The wiki for K12 has some other interesting controversies).

Example: K12

• “On the Morning of August 31, the first day of school, the 345,000 students in Miami-Dade County’s public schools fired up their computers expecting to see the faces of their teachers and classmates. Instead a scruffy little dog in banana-print pajamas appeared on their screens, alongside an error message. “Oh bananas!” read one message from the district’s online learning platform. “Too many people are online right now.”
• “A rudimentary cyberattack had crippled the servers of the nation’s fourth-largest school district, preventing its 392 schools from starting the year online. But even once the district had quelled the distributed denial-of-service attack and a local teen had been arrested for the crime, “Banana Dog” didn’t go away. If anything, the security breach merely obscured for a few days the crippling weaknesses in the district’s plan to move every aspect of its schooling—including a revamped curriculum—onto a platform that had only ever supported half as many students (and never all at once).

Example: K12

The problems were visible at a distance, and teachers saw it coming:

• “The teachers received demo logins to try out the platform, but they didn’t work, and even the trainers struggled to access it, West says. From 8 am until 3:30 pm each day, teachers took notes without once trying the software themselves. “The training was make-believe, it was so, so complex,” says one teacher. “Even our techie teachers were lost.” On Facebook, teachers shared GIFs of dumpster fires and steaming poop emojis in response to the experience.”

​

​

Example: K12

In the end, teachers ended up using a hodge-podge of tools:

• “Miami-Dade’s teachers have now pivoted back to using two apps they’d used in the spring: Microsoft Teams and Zoom. “It’s mostly been sighs of relief,” Perez says. The episode would have been an expensive boondoggle, except for one thing: The district never signed its [15 million dollar] contract with K12.”

​

​

“Three Things We Learned at Khan Academy Over the Last Decade” [Link]

From January 2020 (pre-Coronavirus):

• Thing 1: ”Teachers are the unwavering center of schooling and we should continue to learn from them every day.”
• “At the beginning of the decade, we worked with a handful of early adopters.”
• “I think we all knew, even back then, that technology could one day deliver on the long-held dream of helping teachers to differentiate instruction for every student. At the same time, we knew that technology was nowhere near a silver bullet.“
• “I was humbled by these early teachers’ feedback and the learnings they shared.”

“Three Things We Learned at Khan Academy Over the Last Decade” [Link]

From January 2020 (pre-Coronavirus):

• Thing 1: ”Teachers are the unwavering center of schooling and we should continue to learn from them every day.”
• “Fast forward to 2019. We’re working with teachers across the country now. They’re using technology as an integral part of their instruction and personalizing instruction for a classroom of students with diverse learning needs.”
• “Equally important, they’re never using “technology for technology’s sake.” I, like many teachers, share a deep belief that we should decide how to best meet the needs of students before turning to technology. Not the other way around. Technology can be pen and paper, or adaptive software, or whatever is most helpful for the learning task at hand.”