How Can We Make Office Hours Better?

SIGCSE 2021 Birds-of-a-Feather

Kristin Stephens-Martinez

Duke University

Brian Railing

Carnegie Mellon University

Kevin Lin

University of Washington, Seattle

kevinl.info/office-hours

Agenda

Goal: Learn from our collective experiences running online office hours.

5:30 pm — Introductions

5:35 pm — Breakout rooms to add ideas to these slides for later deep dive.

5:50 pm — Bring everyone back and have a gallery walk through this slide deck.

6:00 pm — Breakout rooms with new slides developing earlier ideas.

6:15 pm — Report back and closing remarks.

6:30 pm — Breakout rooms for networking and smaller conversations.

Room 1

What do you do for office hours? Pros? Cons? Problems?

• Queuing system, where student can go in queue asynchronously waiting
• Pro: Provided better 2 on 1 with TA
• Problem: Some need to triage, so closer to deadline, scope of issue
• Problem: Students swarm the queue when it opens. Also few of them game the system (more of an issue in giant classes).
• Calendly
• Pro: Allows students to schedule 15-minute zoom logins
• Problem: small costs
• Specific office hours for conceptual questions
• Pro: less context switching for TAs, can group students up easier
• Problem: not very well utilized
• ​

Room 2

What do you do for office hours? Pros? Cons? Problems?

• Problems
• Time management
• making sure everyone gets a turn
• Support large #s of students with limited time
• Giving the “right” amount of help (helping without giving away answers)
• Big increase in demand in the remote setting
• Potential solutions
• Single giant Zoom for everyone, with breakout rooms for individual TAs
• Survey people to find out common questions, address them in a group setting or Piazza
• FAQs for each assignment on Piazza
• Use a queue system that requires students to write what they want to ask about
• Group students from different classes that have different assignments but the same topics, to avoid giving away answers

Room 3

What do you do for office hours? Pros? Cons? Problems?

• Multiple TAs in office hours session (zoom) (CS1)
• Pro: worked great when the order of participants was in order of attendance
• Problem: zoom is now alphabetical, so can’t tell arrival time
• Online queue website
• CMU - OHQueue (cmu.ohqueue.com)
• Online - student provides zoom link and hopefully has code already shared
• In person - office hours area - students can work together and get help at the same time
• NCSU - mydigitalhand.org
• UBC - queuing system as well
• IU - write name on the board to keep order
• Set place for office hours
• UBC - TAs book tables for course - set place for office hours
• Problem: sometimes students can’t find the TAs (since TA isn’t looking for student)
• Having space where students can work and hang out while waiting is helpful

What do you do for office hours? Pros? Cons? Problems?

• Time limits
• Trying to make 5 minutes per interaction
• # of office hours request and queue time
• NCSU Seeing more requests and longer wait times
• Flexible - teams/slack
• Problem: boundaries are challenging
• Office hours location
• UBC - make office hours closer to lecture hall location - reduces barriers to attendance
• NCSU - online office hours have reduced barriers for attendance

Room 4

What do you do for office hours? Pros? Cons? Problems?

• Caltech intro courses: common office hour qn:I have a bug, please help
• Pro: effective at resolving student bugs + getting them unstuck
• Problem: overreliance, overwhelming OH during difficult sets
• U.Utah intro: have a queue of students. TA queue. (+1)
• TAs can help people in batches by identifying people with common questions on queue.
• Not scalable day before assignment due.
• Discussion forums
• Seems like some students really prefer office hours (especially in virtual quarter).
• Perhaps lower friction now that students can “just click a link” or change servers. (+1)
• How do office hour work virtually
• Take existing software and run it virtual (discord channels)
• Harder in virtual world to “group people up” Or students to group themselves

Room 5

What do you do for office hours? Pros? Cons? Problems?

• Office Hours chat room for larger class (Soham)
• Pro: great for managing multiple TAs sharing OH
• Problem: very easy to go and work with 1 student for a long time
• By Appointment Only (Steven)
• Start/End lectures by pointing how to help (Lina)
• Ask students what their biggest concerns - not knowing where to get help
• Adjust # of office hours during Project weeks (larger assignments due)
• Use MyDigitalHand (https://beta.mydigitalhand.org/) (+1)
• Standard times -- every evening 6-8pm
• Use same zoom go -link for all office hours
• Google Form for Office Hrs (Travis)
• Scheduled Time and By Appointment Too

Question: How do we know whether our OH is effective?

Names: Steven Bitner, Lina Battestilli,Soham Pardeshi, Travis McGaha , Klas Arvidsson

​

Room 6

Problem: How do I get students to come to office hours?

• Incentives for attending office hours. Give points? Snacks?
• Culture of students’ practices. Some students figure out that the way to succeed in the course is to go to office hours. Student groups have really normalized this message to address the concern that students might think that going to office hours means you don’t know!
• If there are too many students, maybe assignments are too hard! Challenging-ness is a lever for office hours.
• Availability and accessibility of office hours: can students actually attend?
• Tell them: “Because Algorithms doesn’t have a lab component, you can use my office hours as if it’s a lab. I don’t mind if you come here to do your homework. I’ll just put you in a breakout room and just talk to them.” The thing that held them back was needing to have a question.
• It’s hard to catch students who aren’t used to office hours!
• Try to get students in office hours early on in the quarter, maybe mandatory OH just to help them get familiar with the office location

​

Room 7

What do you do for office hours? Pros? Cons? Problems?

• Gather Town (Tom - small classes at Occidental)
• Pro: Closer to the real experience.
• Problem: If it gets too big, things don’t work well.
• Jacqueline CS1 at Toronto (2000+ students, 50 TAs). Flat TA hierarchy.. Timed office hours running 8 AM - 11 PM 5 days a week. Blackboard collaborate ultra. Simultaneous OH primarily focused on 1-on-1 help.
• Pro: Reasonable replacement for in-person office hours room.
• Problems:
• Demand has been extremely high. Informal network of support for basic questions has been replaced with going to TAs.
• How to get TAs spun up to do a good job and give them feedback.
• Repetition of questions (other than just saying go to FAQ) - how do you deal with?
• In person one can utilize the saying out loud “Hey who has X question?” option.
• Alex: You are on a queue. Sufficient number of questions on the queue that are on the same topic, you can get people together for a session (previous semester 10 minute max).

Room 8

What do you do for office hours? Pros? Cons? Problems?

• Purpose of OH
• What students come to OH about?
• How to establish the valid scope of questions
• Attending TA versus Prof OH
• Students prefer one versus the other, and can depend on course / institution
• Triage students in OH
• Managing limited time
• Polling students about times for OH

Round 2

Room 1

What do you do for office hours? Pros? Cons? Problems?

Some problems different at small vs. large schools

• Office hours can be underutilized at small schools
• But still can have issues with repeated questions, high demand near deadlines

Discussion forums (e.g. Piazza, Discord) can reduce demand

Common problems

• Students often don’t start early enough
• Some students don’t know how to begin, come to office hours to get started
• Students often don’t know how to get unstuck (e.g. how to debug)

How to encourage students to use both TA and professor office hours

• Suggestion: integrate them so that students don’t see a difference (e.g. hold block office hours, with TAs and professors there concurrently or trading off)
• Help students to become comfortable talking to professors, humanize them

Room 2

What do you do for office hours? Pros? Cons? Problems?

​

• Robbie - (500-student course)�- 30-min meeting with a TA� - hard to get conversations started�- Normal queuing system & Discord� - some students don’t have as much participation� - bring everyone with similar question in, but involves sharing code
• Jacqueline - Pre-assigned breakout rooms in lecture
• Majority of students don’t seem to interact with each other in breakout room :-(
• Other students frustrated with the students who don’t talk
• David - flipped classroom
• Do some HW in class, as opposed to pure-lecturing
• Leave assignments open-ended, so there can be different code implementations so students would need some understanding�

What do you do for office hours? Pros? Cons? Problems?

​

How to autograde randomize code?��Some “novelty” between first adapting to online class vs doing it for a few quarters now (can be an “appeal”?)��Discussion boards�- private questions might not work well for large-scaled class ���

Room 3

What do we do for OH

• Big Zoom for students, TAs are in breakout rooms
• Latest week: Peak students 39, 157 sessions across 5 days
• Online tool system
• How do we get students to come to OH?
• Tried calendly for students to sign up for appointments

Room 4

Running Office Hours

• For how many students/for what courses is a queue appropriate?
• 5 mins per student
• Can you even solve/help in 5 mins?
• Students come prepared with a bug
• Get them somewhere to move on to
• Groups for conceptual questions
• 3-4 students
• Student squatters
• Prioritize students who have not been to OH before
• How do you help students and not overhelp squatters?
• Not allowed to join before OH time
• OH for specific assignments

Room 5

Office Hours Discussion

• Homebuilt office hours queue (Berkeley)
• Both in-person and online
• Easy to put in tickets and pull students off the queue
• 1000+ person courses
• Challenges - getting students to work together (both tooling and departmental policies)
• Integrates with course tools
• https://oh.cs88.not.cs61a.org (Code Repo)
• Challenge - getting students to ask good questions
• Questions get worse towards deadlines
• Identifying if students are fishing for answers/office hours questions
• Assignments to encourage questions
• Challenge - students don’t attend office hours when struggling with course
• Secondary challenge - identifying students who are struggling
• Courses with labs - lightweight oral lab check-offs - 2-3 verbal questions if “get” concepts

Office Hours Discussion

• Identifying struggling students
• Look at major grade differences between HW X and HW X+1
• Create small group conceptual office hours to discuss prior office hours
• Extra sessions outside of class - go through example problems that might be like something on the test
• Or have TAs do this!
• Tutors - junior TAs
• Office hours sessions with multiple tutors to support each other
• Instructor Office Hours vs TA Office Hours
• Students may be intimidated by instructors; TAs are near peers
• Office Hours time
• 250 students with 30ish office hours
• 130 students with ~30 office hours
• 280 students with 20 office hours (there’s also labs that are pretty much office hour-ish)
• 60 students with 30-35 office hours
• Add extra office hours so TA can make hours (more pay!)

​

Room 6

• Seems like there are extremes - overutilization or underutilization
• What to do when the student is really lost? Where do you start?
• Context matters
• Lightly prod the student to start talking about what they are doing

Room 7

Metrics for evaluating office hours? What’s success?

“The people who come to office hours seem to follow a zipf distribution.” How do we reach students who really need to come but who aren’t there? In big classes, there are plenty of students who do come but worried about those who don’t and would really benefit. Maybe effectiveness is partially about coverage: of the students who are “struggling”, how many are using office hours?

In office hour queue systems, can have TAs and students rate each other after an interaction. Just going by student feedback is not the best: TAs that give out the answer might be highly appreciated but not ‘effective’ per se. Implicit bias.

Qualitative evaluation of student questions over time.

Lower wait times (see the next slide, “How to address online wait times?”).

How to address online wait times?

• Some days there are 3-hour wait times. Worse for online office hours.
• Possible diagnosis: roadblock students—some TAs don’t want to disengage from helping a student, which backs-up the queue. Even worse: the next student wants extra time since they waited so long.
• Fix: Reduce the number of hours offered per week to increase staffing per hour.
• Use data to select hours and communicate this approach with students.
• Interesting data point: 50% of office hours resources used by 5% of class.
• Who’s using office hours? Helping students feel welcome to come to office hours!
• Discussion board helps.
• Can calibrate difficulty with Robert Talbert’s two questions:
• What’s the difficulty of the course this week?
• How supported do you feel this week?

Room 8

OH better? Pros? Problems?

• Queue Management
• Managing time - keeping time to some constraint
• Finding an actionable point - students can make progress
• When queue has capacity -- and it closes if too many students.
• Consistency across TAs - time spent
• Making queue waiting more productive
• Common waiting room
• Triaging student issues
• Deadline load
• Student demand increases near deadlines
• Some courses lower OH near deadlines
• Some courses increase OH near deadlines
• Students not coming to OH
• What is the barrier?
• What can we do to reduce OH demand?

General discussion notes

• Student says "It doesn't work"
• Have a pinned Piazza post for how to ask questions
• Template. If students don’t provide enough information, ask them to do so first! Can use a Piazza bot to do this to avoid putting blame on a particular TA. How are we teaching students to think about this?
• Steps to reproduce
• What happened
• What expect to happen
• Have students share a failing test case
• Gauging student question: “Oh, will this work?” can be a sign of “fishing” for answers.
• Have students question their own work: metacognition.
• Examine the positionality/framing/bias that we bring to how we engage with students.
• https://edstem.org/ has discussion post templates and bot support.

General discussion notes cont.

• How to help students frame questions clearer? How to get them to ask “better questions”?
• Modeling the process of starting with an ambiguous question to landing on a more articulate question for students (the template in Ed that Kevin shared is an automated way of doing this)
• Also I think putting the onus on first time programmers to ask better questions is not the way to go (also more deficit minded)
• This question is really dependent on the level of the course. It’s difficult for students (particularly those in lower level courses) to ask articulate questions because a student might not have any idea of what’s going on.
• Consider assuming (say, for the first half of an intro course) that no student is fishing and really engage with students on where they are instead of telling them to just give more information on the question.
• It’s helpful to not have “is this student fishing for answers” in your head b/c I think it can jade your perspective on students’ questions.

Possible discussion post template

Which task are you working on? Describe the task in your own words.

TODO: Fill this in!

What do you think is the problem? If there's an error, what suggestions have you found by searching on the web? If the code isn't working as expected, what lines of code do you think are responsible? Include relevant error messages or code snippets.

TODO: Fill this in!

What have you tried so far to resolve the problem?

TODO: Fill this in!