How to be a great (CS) teacher

Andy Ko, Ph.D.

University of Washington

The Information School

Why me?

I’ve been teaching for 20 years

Tenure-track faculty for 9 years

I teach HCI, software engineering, research methods, capstones

I’ve spent the past 5 years reading deeply in learning science and education research

I now do research on how people learn programming and computing

Andy Ko — faculty.uw.edu/ajko

If you don’t understand (or believe) something I say, please interrupt

Share your your CS education story with your neighbor

(2 min)

You are not your students

You’re the most capable student you’ll ever have (if you’re lucky)

Most of your students will need much more help than you did

​

Andy Ko — faculty.uw.edu/ajko

Everyone can learn CS

Every effort to predict successful learning is dominated by:

• Prior knowledge
• Motivation
• Quality of practice

There is no evidence of a “geek gene” that predicts successful learning of computer science. It is not innate.

Patitsas, E., Berlin, J., Craig, M., & Easterbrook, S. (2016, August). Evidence That Computer Science Grades Are Not Bimodal. In Proceedings of the 2016 ACM Conference on International Computing Education Research (pp. 113-121). ACM.

Andy Ko — faculty.uw.edu/ajko

If students are failing your class...

It’s because:

• Students don’t have the prior knowledge you expected
• They aren’t sufficiently motivated (by you or themselves)
• Your class lacks sufficient high quality practice

Andy Ko — faculty.uw.edu/ajko

So what?

Why worry about failure?

Reflect on how you got your job:

• Someone helped you acquire knowledge
• Someone helped motivate you, give you confidence
• Someone helped you structure your practice

Now that someone is YOU. If you fail, you’ll be robbing the world of the next you :)

Andy Ko — faculty.uw.edu/ajko

How can you promote successful CS learning?

Learning comes from “deliberate practice”

Consists of four things:

1. Sustained motivation
2. Tasks that build new knowledge from prior knowledge
3. Immediate personalized feedback to build correct knowledge
4. Repetition above the above

If any of these are missing, learning doesn’t happen.

Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological review, 100(3), 363.

Andy Ko — faculty.uw.edu/ajko

Motivation: your mindset

Many teachers (possibly some of you) don’t believe that all their students can learn.

This is called a fixed mindset.

When teachers have fixed mindsets about their students’ intelligence, students come to believe they can’t learn either, and they stop being motivated.

You need to adopt a growth mindset (the belief that intelligence comes largely from deliberate practice, not genetics).

Dweck, C. S. (2008). Mindset: The new psychology of success. Random House Digital, Inc..

Andy Ko — faculty.uw.edu/ajko

Motivation: student mindset

Even if you have a growth mindset, many of your students won’t.

You have to convince them that they can learn.

• Encourage them
• Prove to them they can learn by giving them practice they can succeed at
• Show them that others also struggle, mitigating imposter syndrome

Andy Ko — faculty.uw.edu/ajko

Motivation: always be selling

To be intrinsically motivated to learn what you’re teaching, students need to believe that the content is relevant to their interests, goals, and identity.

That means you need to know their interests, goals, and identities.

• Learn about your students as individuals (e.g., their names, goals, interests)
• Link every concept in class to their interests, goals, identities (e.g., start class with why you’re teaching something, then teach it)

E.g., teaching sorting functions? Explain to them why it relates to their ambition to work at Google, their identity as an engineer, or their interests in optimization.

Hidi, S., & Renninger, K. A. (2006). The four-phase model of interest development. Educational psychologist, 41(2), 111-127.

Andy Ko — faculty.uw.edu/ajko

With your neighbor: how have your best CS teachers motivated you in class?

(2 min)

Tasks: prior knowledge

Devise homeworks and other practice that build upon your students’ knowledge

To do this, you must first measure what they know

Prerequisites are poor indicators of prior knowledge (because grades are usually invalid, unreliable measures of knowledge)

Instead, at the beginning of each class, give a pre-test:

• What concepts in your class do they already know?
• What concepts do you expect them to know, but they don’t?

Adjust what you teach based on their prior knowledge.

Andy Ko — faculty.uw.edu/ajko

Tasks: individual differences

You’re likely to have students with widely varying prior knowledge.

Design practice (e.g., homeworks, activities) that account for this:

• Give challenge tasks to students with more prior knowledge
• Give extra instruction to students with less prior knowledge

For example, design homeworks with multiple levels of success, all of which receive credit, and offer time for students to get help achieving all levels of success.

​

Andy Ko — faculty.uw.edu/ajko

Tasks: scaffold irrelevant hard parts

In CS, problems and solutions are often highly coupled

It’s hard to separate the part of a problem or solution you want a student to learn from the other hard parts of the problem they haven’t learned about yet

Scaffold the irrelevant hard parts—remove those details or give partial solutions, and slowly take the scaffolding away.

Base scaffolding on students’ prior knowledge.

Andy Ko — faculty.uw.edu/ajko

With your neighbor: what was your favorite CS homework from your education? Why?

(2 min)

Feedback: formative versus summative

Deliberate practice requires detailed, personalized, immediate feedback about what someone did right and wrong in their practice.

A “C+” on a quiz is not detailed or immediate: it only punishes a student for failing to learn (or for your failure to teach), reducing their motivation.

• Focus on iterative formative feedback that grows ability through success
• Use summative feedback sparingly; it’s delayed and discouraging

For example, before a (summative) midterm that determines a grade, give students several (formative) practice midterms and give detailed feedback about their work.

Andy Ko — faculty.uw.edu/ajko

Feedback: as soon as possible

Immediacy of feedback is critical: the sooner they receive it, the better their learning.

Qualitative > quantitative because it explains mistakes rather than just signalling them

Provide feedback on homeworks as soon as they are submitted, giving detailed qualitative feedback about everything they did right and wrong.

Incentivize students to show their work so you can critique their work. Don’t just tell them whether they were right or wrong.

Andy Ko — faculty.uw.edu/ajko

With your neighbor: what’s the best kind of feedback you’ve received from a CS teacher?

(2 min)

Repetition: promote mastery

Most classes are designed to only allow a student to practice something once

Design your class so that students can practice something multiple times, across a course, until they master the knowledge

Don’t just give them one problem to solve, give them multiple isomorphic problems to solve.

And when they get some of those problems wrong, keep giving them practice until they get it right.

Andy Ko — faculty.uw.edu/ajko

What does this look like in practice?

My 1996 STATS 101 class in college:

• 200 students
• 10 teaching assistants
• 20 topics, sequenced
• 10 isomorphic quizzes per topic, 10 attempts on each topic
• Students studied for each quiz, took the quiz, got feedback from the TA about mistakes, and retook the quiz until getting a perfect score
• Students passed the class if they mastered all 20 topics within the 10 week quarter; most received a 4.0.

It was the best, most rapid learning that I’d ever experienced (and it used no faculty)!

Andy Ko — faculty.uw.edu/ajko

Why great teaching is hard in higher education

Scale

When you teach 50+ students

Personalizing feedback is hard to scale

Knowing students’ individual interests, goals, and identities is hard to scale

It is hard to design for widely varying prior knowledge

Andy Ko — faculty.uw.edu/ajko

Structure

It doesn’t account for variation in motivation and prior knowledge.

Required courses will necessarily have students with lower motivation to learn what you’re teaching

Quarters and semesters artificially constrain the time students have to practice

CS courses often have too many learning objectives to meet, preventing adequate repetition

Andy Ko — faculty.uw.edu/ajko

Incentives

They don’t promote learning.

Grades warp student motivations; they become obsessed with optimizing GPAs instead of optimizing learning

Your job description says teaching is your second priority behind research

Designing courses is hard; once you have a design, it’s hard to motivate yourself to redesign it if it’s not working

It’s hard to know teaching is not working.

If students are failing to learn, you keep your job anyway.

Andy Ko — faculty.uw.edu/ajko

Context

Students have to learn multiple things at once

Students are taking other courses they may care more about

Students have jobs, relationships, debt, fear of the future, and other stress to manage

No matter how well you motivate them, they may necessarily need to give their attention to these things instead of your class

Andy Ko — faculty.uw.edu/ajko

Why great teaching can be easy in higher education

Great teaching can take less time

Lectures are terrible for learning, take more time to write, maintain, practice, and perform than active learning methods.

Personalizing learning requires less preparation because you’re generating instruction in real-time, not designing it in advance. It just requires your attention and expertise.

Andy Ko — faculty.uw.edu/ajko

Teaching is your biggest impact

Let that motivate you.

Your research might change the world. It might not.

Your teaching will definitely change the world, shaping how tens of thousands of people will think, act, and create in the world.

Do a great job and they’ll fund your endowed chairs, your buildings, and your research.

Andy Ko — faculty.uw.edu/ajko

Computing education research solves the hard parts

Conferences and journals like ACM ICER, ACM SIGCSE, and ACM TOCE are full of rigorously evaluated insights about the best way to teach CS content of many kinds.

Read the proceedings, and let us do the hard work for you

Better yet, hire one of my Ph.D. students into a tenure-track position, and they’ll teach you.

Andy Ko — faculty.uw.edu/ajko

An example

INFO 461

Cooperative Software Development

A small class full of aspiring developers

Undergraduate (seniors)

Elective

35 students

Prerequisites: CS1, CS2, algorithms, databases, web development, HCI

Andy Ko — faculty.uw.edu/ajko

Required readings

http://faculty.uw.edu/ajko/books/cooperative-software-development/

Instead of lectures, I make them read short chapters covering the same material

In-class free-response quiz on key concepts to demonstrate understanding

“Think-pair-share” after quiz, then we discuss right answers

This identifies and corrects misunderstandings, while helping me find flaws in the readings.

Andy Ko — faculty.uw.edu/ajko

Selected readings

Your research!

Students choose a paper from a “further reading” list

They write a summary and discuss what was interesting

In-class, they discuss what they read in groups of four

I ask for volunteer to summarize and share what was interesting

I respond, connecting their thoughts to the broader topic of the class

Andy Ko — faculty.uw.edu/ajko

Midterm

Incentivizes careful reading

Helps me measure whether I successfully taught the material.

If everyone gets 100%, I succeeded.

(I failed this quarter the average was 85%, so I spent time in class helping them understand what they got wrong, then used this understanding to improve my book)

Andy Ko — faculty.uw.edu/ajko

Homeworks scaffold planning

Personalized feedback for every step. I use 1 hour of class time to work with each team individually to scaffold, critique, and guide their planning.

Design your organization (4 points)

Design your software(4 points)

Derive requirements (4 points)

Derive specifications (4 points)

Design your process (4 points)

Derive a verification plan (4 points)

Release (10 points)

Evaluate (2 points)

Patch (2 points)

Reflect (2 points)

Andy Ko — faculty.uw.edu/ajko

Release

They ship a 1.0 in three weeks

They’re graded on defects detected by their peers

This incentivizes careful execution of their verification plans, careful planning of architecture, careful feasibility assessment

All of my instruction in the final 3 weeks is answering their questions in Slack and in class, 100% personalized, semi-immediate feedback.

Andy Ko — faculty.uw.edu/ajko

Does it work?

More than half rely on me for mentoring and resources (which I provide in-class)

The students are behaving not like new hires, but like developers with 1-2 years of theoretically-grounded experience

I only spend 4 hours in class and 2 hour a week providing feedback (the TA evaluates readings).

Andy Ko — faculty.uw.edu/ajko

Deliberately practice your teaching

Stay motivated

Remember that teaching is your biggest source of impact

You can change a student’s life with just one word of encouragement or a critical insight about their trajectory

Andy Ko — faculty.uw.edu/ajko

Challenge yourself incrementally

Don’t try to learn five new teaching strategies all at once.

Teach classes that leverage the teaching skills you have

But always challenge yourself to learn new teaching skills.

Andy Ko — faculty.uw.edu/ajko

Seek feedback

Reflect on your teaching

Ask students for feedback regularly

Have your colleagues observe your teaching and provide feedback

Use your students’ success as a measure of your success (everyone should pass)

Andy Ko — faculty.uw.edu/ajko

Repeat yourself

Teach the same courses until you master them

Then, raise the bar

Andy Ko — faculty.uw.edu/ajko

Don’t focus on “techniques”

Focus on the execution of techniques from a deliberate practice perspective.

Strong evidence against lectures (unless they are combined with peer instruction)

Strong evidence for “doing” (unless students receive no feedback)

Strong evidence that tests improve learning (but only when feedback is qualitative, formative)

Andy Ko — faculty.uw.edu/ajko

Questions?

SUMMARY

Learning requires deliberate practice:

• Motivation
• Tasks that build on prior knowledge
• Immediate personalized feedback
• Repetition

How?

• Evaluate your course designs for deliberate practice
• Borrow ideas from computing education research
• Regularly reflect on your practice

If you do, you’ll impact thousands of people and save time.

WANT TO LEARN MORE?

Read How People Learn: Brain, Mind, Experience, and School:

https://www.nap.edu/read/9853/chapter/1

Read my blog:

https://medium.com/bits-and-behavior

Read the computing ed research literature:

http://faculty.uw.edu/ajko/cer