Instructor: Douglas Fisher

Course: Machine Learning, Introduction to Databases

University: Vanderbilt

Table of Contents:

Resources

Overview of classes

Database Class, Spring 2012:

Out-of-class activities

In-class activities

Database Class, Spring 2013:

Out-of-class activities

In-class activities

Additional work required

Student response

Machine Learning Course, Spring 2012:

Out-of-class activities

In-class activities

Machine Learning Course, Fall 2012

Out-of-class activities

In-class activities

Overall Flipped Experience

Obstacles encountered

Things that went well

Things to do differently

Student reactions

Amount of extra work?

Resources

Wrapping a MOOC: Student perceptions of an experiment in  blended learning

Overview of classes

Doug Fisher of Vanderbilt flipped his undergraduate database course with Jennifer Widom’s Introduction to Databases, and his graduate-level machine learning course with Andrew Ng’s Machine Learning. Fisher taught two flipped database courses (Spring of 2012, Spring of 2013), and two machine learning courses (Spring of 2012, Fall of 2012). Fisher used the content freely available on Stanford’s website for his Spring 2012 offerings, and the content on the Coursera platform for subsequent offerings.

Database Class, Spring 2012:

Out-of-class activities

In addition to using Jennifer Widom’s Introduction to Databases lectures, Fisher also adopted Widom’s textbook (co-authored by Jeffrey Ullman) to keep the course material consistent. Outside of class, students completed assigned readings and watched lectures. The class met on Tuesdays and Thursdays; every Tuesday morning, students were given a 15-minute quiz on the out-of-class materials.

Students also worked on two large projects in the course, both team-based. The first involved reverse-engineering a database for a dorm energy monitoring site, the second involved designing a database that would support a social network of educational material.

In-class activities

Fisher would typically start class by talking briefly about the lectures and assigned readings before transitioning into small group problem solving activities (e.g. “analyze some design as a class,” “write queries in small groups,” etc.). With each of these activities, Fisher aimed to help his students “exercise the concepts that they learned in the lectures and reading.” As noted above, students would be given a 15-minute quiz on the out-of-class material each Tuesday morning in class.

Database Class, Spring 2013:

Out-of-class activities

These activities were very much the same as in the Spring 2012 offering, to include roughly the same assignments and the same two large team projects; each team was composed of three students.

The weekly quiz that Fisher gave in class on Tuesdays in Spring 2012 was taken out of class in Spring 2013. In 2013 students downloaded, completed, and submitted the quiz from the course site before the first class meeting of the week.

In-class activities

Because the Spring 2013 course was offered at 8:10 in the morning, Fisher allowed each of his students to sign up for small “study groups” that met at different times on Tuesdays and Wednesdays of each week. Fisher scheduled five of these study groups (one of which was the original 8:10 am class time) based on student availability as indicated in a Doodle poll. Fisher created a number of exercises for the study groups which they’d solve and upload to the class discussion boards to share with their peers. To keep track of attendance, groups were required to list each of the names of the attending members in their study group submission.

Fisher would often visit each study group for only a few minutes at the beginning of each session. To accommodate his students’ schedules, Fisher allowed each student to miss up to five meetings; anything beyond that would result in docked points.  With this system, Fisher reports that he got nearly 100% attendance, and that the average number of missed meetings was around two.

Students were also organized into teams of roughly three for purposes of the two team-based projects. Fisher met with two groups at a time for one hour each week, with four weekly meetings in total; all four of these small team meetings with Fisher were scheduled on Wednesday or Thursdays. In these meetings, each team would present on the status of their project, after which Fisher, a TA, and other students would provide feedback.

Additional work required

Fisher reports that he spent slightly more time with students than he had spent in previous offerings of his course (4 hours vs 2.5 hours/week).

Student response

Fisher reports that course ratings have gone up as a result of his flip. The Center for Teaching at Vanderbilt also did an evaluation with his students, mid-semester and found that the most preferred this format over the traditional model.

Machine Learning Course, Spring 2012:

Out-of-class activities

Before meeting for class on Tuesdays and Thursdays, students were required to watch Andrew Ng’s lecture videos in addition to a journal article assigned by Fisher. When possible, Fisher selected journal articles that complemented the concepts in Ng’s lectures, though Fisher often wanted to cover very different machine learning topics that were not covered by Ng’s lectures.

Students also completed three projects, two of which were re-implementations of and experiments with machine learning systems that were described in two of the assigned journal readings, and a third end-of the semester project of the student’s own design. Students turned in their computer programs and experimental results for the first two projects, and gave an end-of-the semester presentation to the class for their final project.

In-class activities

Class time focused more on the journal paper assigned than Ng’s lectures. However, Fisher emphasized moments of overlap and synergy between Ng’s lectures and the assigned readings as much as possible. Students were given a quiz every Tuesday at the start of class that covered both the reading as well as Ng’s videos to ensure that they were doing the out-of-class work. There was one midterm exam as well. There was no final exam, but the assigned time slot for the final exam was used for presentation of the end-of-the-semester projects noted above.

Machine Learning Course, Fall 2012

Out-of-class activities

In the fall of 2012, Fisher taught another section of Machine Learning as an overload course. The bulk of the student work took place on the Coursera platform: students were expected to complete Ng’s course in it’s entirety and present a Statement of Accomplishment upon their completion. In addition to completing Ng’s MOOC, students were assigned weekly reading and expected to complete one project that consisted of building a machine learning system. At the end of the course, students presented their projects to the class. This final project corresponded closely to the final project in Spring 2012, but there was no correspondence in the Fall 2012 course to the first two projects of the earlier Spring course.

In-class activities

Fisher reports that this course was far less time-intensive than his previous offerings because “most of the work was being done by Coursera,” Fisher met with his class only once a week for 90 minutes. During these sessions, the class primarily concentrated on the assigned journal readings.

Overall Flipped Experience

Obstacles encountered

For the machine learning courses, Fisher reports that the most difficult part of flipping his classroom was often in finding connectedness between the journal articles he assigned and Ng’s pre-recorded lecture videos on Coursera. In some cases, there was simply no substantive overlap between the journal articles and Ng’s lectures, and that this was by design -- Fisher thought that coverage of this additional material was important. He said that this was particularly problematic in the Fall 2012 offering of his Machine Learning course because Ng’s class “wasn’t a great fit” to his in-class activities, and because he was spending much less time discussing the Ng material in class in Fall 2012 than in Spring, largely because of the less time spent in class (90 minutes per week in Fall 2012 versus 150 minutes per week in Spring 2012). Consequently, “students had more of a sense of schism or disconnect,” in Fall 2012 and they “seemed to be more concerned about this than we thought.” He states that the “degree of coupling” is something that instructors should really focus on when planning their flipped classrooms.

In the Database classes of Spring 2012 and 2013, flipping came much more naturally -- it was relatively straightforward to design active learning materials for the flipped classes, though actually preparing such material took substantial time in Spring 2012, much of which Fisher reused in Spring 2013.

In the Spring 2013 database course, some students complained (at the mid-semester Center for Teaching assessment) that in the study groups that Fisher had scheduled for Tuesdays and Wednesdays of each week, they “didn’t know if they were going down the wrong path” when they were doing the exercises with their study groups. To address this, Fisher started posting exercise keys that allowed students to calibrate their progress and self-correct when needed. Moving forward, Fisher would like to prevent situations where the faster students arrive at answers more quickly and give these answers to the slower students in a way that precludes learning by the slower group.

Things that went well

Fisher reports that both course and instructor evaluations improved significantly for the database and machine learning courses in 2012 and 2013, as compared with versions of these courses that were offered before 2012. He believes that the “floor came up,” in that the worst performing students from the flipped database courses were better than the worst performing students of earlier (pre-2012) non-flipped classes. Fisher has since flipped other of his on-campus courses (e.g., artificial intelligence), and that this experience has gotten him “more excited about teaching than I’ve been in 25 years.”

Things to do differently

Fisher said he has learned that the instructor has to do a lot more work in advance when flipping the classroom. When using licensed content from other professors, Fisher advises that the instructors know the material well in order to find moments of overlap between the activities/assignments they design for the class, and the licensed content. For future iterations of his course, Fisher reports that he will continue his recording some of his own videos, which he barely made a start of in late 2012 and into 2013, to complement the lectures and assignments in the licensed content.

Fisher also reported that he’d like to continue to explore the “distributed model” of allowing students to meet at times that are most convenient for them. He envisions that his students will be able to form their own study groups and meet regularly outside of class.

Student reactions

Aside from complaints in the Fall 2013 machine learning class about the disconnect between the in-class discussions and lecture videos, and in the Spring 2013 database class about the lack of guidance in the study groups during the first half of the semester (see above for more information), Fisher reports that students were very positive about the flip across each of the classes.

Amount of extra work?

Fisher reports that the 2012 database class “might have been a little more work” because of preparing the active learning materials, however he notes that this was “only an initial investment,” and he saw time-savings with respect to this activity in 2013. In the 2013 database class Fisher spent more in-class time with students (4 hours versus 2.5 hours in earlier offerings) because of his four weekly meetings with project teams, but that overall the Spring 2013 offering was comparable to pre-2012 non-flipped offerings of database. Fisher believes that he spent “probably one quarter of the time” in his latest Fall 2012 offering of Machine Learning compared to previous offerings of the course.