2019 SFC Computer Science:
Conference Proceedings and Recommendations
Committee Members
Adam Blank, Claire Ralph, Chris Umans, Thomas Vidick,
Vivek Bharadwaj, Victor Chen, Karena Chow, Dessie DiMino,
Zachary Domanico, Ethan Pronovost, Anirudh Mathukumilli, Vaishnavi Shrivastava
The undergraduate Student-Faculty Conference (SFC) is a biannual event organized by the ARC. Committees of students and faculty from each option spend a term discussing issues in the department as well as potential solutions, and present their findings at a conference with both students and faculty present.
The 2019 SFC was held on March 8, and there were four topics of focus: a presentation of statistics and demographic information about CS undergraduates, changes to the undergraduate curriculum, improving research opportunities for undergraduates, and academic integrity violations in the department. This document contains information from the survey conducted by the committee, the recommendations put forth at the conference, and the ensuing discussion between the committee and other students and faculty.
The committee conducted a survey of undergraduate CS majors and minors asking questions about undergraduate curriculum, preparedness for industry work, preparedness for graduate school, and other aspects of the option. Aggregate statistics from the survey, as well as the department, are listed below; the full statistics can be found at the end of this report.
Response Rate and Demographics
According to statistics from the CS department, the undergraduate CS option consists of 240 students in total: 81 sophomores, 82 juniors, and 77 seniors. 39% of these students identify as female.
108 CS-affiliated responded to our survey, comprising 92 majors and 16 minors, a response rate of 45%. Roughly a third of students from each year of study (sophomore, junior, and senior) responded to the survey.
98 students self-reported their gender, and 48% of respondents identified as female. 92 students self-reported their race. 58% of respondents identified as Asian, 27% identified as white, 6.5% identified as hispanic or Latino, and 2% identified as black; the remainder identified themselves as mixed race or preferred not to disclose.
66% of those polled entered Caltech with the intention of being a CS major / minor.
Undergraduate Teaching Assistants
60% of survey respondents reported that they had TA’d a class for the CS department at some point, larger than the true fraction of teaching assistants in the undergraduate population. The overrepresentation of teaching assistants in the survey indicates a potential response bias, skewing the responses to certain survey questions. For example, 85% of teaching assistants who responded to the survey believed that they could receive at least one letter of recommendation from a CS faculty member, while only 29% of non-teaching assistants believed that they could receive at least one recommendation letter.
Survey results and members of the committee indicated concerns that some undergraduates have greater access to teaching assistants based on factors such as their undergraduate residence. The committee recommended that the process of becoming a teaching assistant for the department be standardized and better publicized.
During the conference, attendees noted that a web form all polling students who wanted to be teaching assistants had already been set up last year. The form was mailed to all CS undergraduates once in the Spring of 2018, but has not been sent since; several faculty and students were unaware of its existence. The committee recommended that the form be sent more frequently. The CS option representative is in charge of publicising the form a couple weeks before the end of term listing the classes that need TAs. If a class requires specific skill sets for TAs then the professor should add their contact information so that students may reach out to them.
Attendees also discussed at the conference whether the prevalence of undergraduate teaching assistants was viewed as an asset or a burden for students. Student responses largely indicated that the experience of being an undergraduate teaching assistant had been valuable, especially on resumes. Some faculty noted that classes such as CS24 and CS2 had experienced difficulty in finding adequate numbers of teaching assistants recently; students responded that the difficulty stemmed from the small pool of qualified teaching assistants and the limitation that students can realistically only TA a single class every term. The committee hopes that standardizing the process of becoming a TA will increase the pool of TAs and demonstrate the level of interest in TAing.
The committee announced to students and faculty of the following changes in the curriculum for CS majors:
The committee discussed responses to a survey question that asked students to report courses that are not currently offered by the department that they would like to see. Responses indicated that a compilers course is (and has historically been) the number one most requested course since the last SFC meeting. It is not currently being offered due to lack of an instructor and course material, although Michael Vanier is making steady progress towards the offering.
The committee also discussed the hard-4 requirement for CS majors (the stipulation that one of the 72 units of advanced CS electives must contain one of CS122, 124, 139, or 151). The requirement was originally introduced after the 2015 SFC with the aim of ensuring that students take one of two intensive project-based classes (122 or 124) or one of two theory-intensive classes (139 or 151). Several students in the committee expressed concern that CS122 (a class in database system implementation) was widely considered to be an “easier” option that the other three courses, and that enrollment in 122 had spiked as a result (in comparison to its counterpart project-based class CS124; historically, fewer students decide to take the theory-based classes according to TQFR data).
While enrollment in CS122 and CS124 have been roughly comparable in prior years (roughly 33-37 students in each class), enrollment for the 2018-2019 school year indicated that only 28 students took 124, while 63 students took 122, nearly doubling the enrollment from prior years. The committee discussed adding courses to the menu for this requirement, suggesting CS143 as a potential class that met the rigorous criterion for the requirement. Some faculty suggested that CS150 and CS 144 might make suitable additions to the list.
19.4% of all students surveyed planned to attend graduate or professional school. 51% of students believed they could receive at least at least one recommendation letter from a CS faculty member (from a total of 85 responses). Faculty at the conference suggested that a more relevant figure would be the percentage of senior students who believed they could receive at least one letter of recommendation from CS faculty. Of 30 senior survey respondents, 23 responded to the question; of those who responded, 61% of seniors believed they could receive at least one letter of recommendation from a CS faculty member. As noted previously, teaching assistants were far more likely than the general population to report that they could receive at least one CS recommendation letter.
97 students responded to a question on the survey asking them to rate agreement / disagreement with the following statement: “Finding research opportunities is more challenging in the CS major as opposed to smaller majors. 42% of respondents agreed either slightly agreed or strongly agreed, 21% disagreed, and the remainder were neutral. 28% of all survey respondents had undertaken research with a computer science faculty member, and 54% of students have participated in a SURF in some department that required a significant coding component.
Committee members and one faculty member at the conference expressed a concern that a subset of students may be working on “coding SURF’s” that do not provide adequate opportunity to contribute original ideas. 59 students responded to a survey question asking them to rate the balance of work during their SURF as either “mostly coming up with original research ideas” or “mostly coding according to a specification provided by a supervisor”. 52% of the 59 respondents believed that the balance tipped towards coding according to a given specification, while 23% believed that the balance tipped towards coming with original ideas; the remaining 25% believed that their work was balanced between the two extremes.
The problem is exacerbated by the growing size of the CS major and the increasing number of students seeking research in subfields such as machine learning. A faculty member on the committee who has worked closely with students in the department indicated that the number of requests for research in ML has recently outstripped the availability of projects and potential mentors.
The committee recommends two potential solutions to offer undergraduates more research experience:
At the conference, one student expressed a concern that freshmen seeking SURF projects may be hampered by a lack of experience and a “marketable” skill-set; the problem of connecting students who do not yet have a mature CS background with research work that is not menial and suited for their skillset merits further discussion. One faculty member also asked whether the CS department had effectively connected computer science students with research jobs at JPL; the committee’s survey did not ask any questions related to JPL. JPL may be a richer source of meaningful research opportunities, and the topic merits a follow-up.
The growing size of the CS option has been accompanied by an increase in the number of honor code violations in computer science classes. The Board of Control (BoC), responsible for investigating cases of academic dishonesty for undergraduates, reports that the share of cases in CS classes was 30% and 15%, respectively, in the 14-15 and 15-16 school years. The share of CS cases was 75% in the 16-17 school year and has jumped to 85% of all cases as of 3/8/19 in the 2018-2019 school year. Furthermore, the BoC qualitatively reports that more convictions have been the result of overt offenses, such as copying code from repositories or solution sets from previous years, rather than over-collaboration or misunderstandings between students. The trend of an increased caseload in CS has been observed over multiple years, and no single class or any particular year is an outlier. Core CS classes, which routinely have enrollments of more than 100 students, constitute much of the problem.
Year | Share of CS Cases |
2014-15 | 30% |
2015-16 | 16% |
2016-17 | 75% |
2018-19 (ongoing, as of 3/8/19) | 85% |
Table 1: Share of Computer Science BoC Cases over time. All statistics are courtesy of the 2018-19 BoC chair, Chris Johnstone. Note that the number of cases in CS classes cannot be computed by simply multiplying the share of CS classes by the total number of BoC cases in any given year (reported yearly in the Honor Code handbook) due to the fact that classes are often cross-listed between departments. The share of cases for a department X is computed by computing a “score” that is incremented by 1 for every class solely within department X, by ½ for every case cross-listed between X and another department, by ⅓ for every class cross-listed between X and two other departments, and so on. The score is normalized by the total number of cases to get the share per department, and the sum of shares across all departments is 1.
The committee identified the following as potential causes of the problem:
The committee conducted interviews with members of the CS faculty (and one member outside the department) during their investigation of the issue. While most faculty were happy that the centralized BoC handled cases of dishonesty, instructors noted that the process of detecting, documenting, and reporting cases could consume an inordinate amount of time and resources, often with poor results. Faculty expressed concerns about the lack of transparency by the BoC in the outcomes of cases brought by instructors (as well as the lack of aggregate statistics specific to the CS department), as well as some frustration that light nullification and protection decisions by the BoC may not adequately de-incentivize cheating.
The committee’s recommendations include revising collaboration policies within the department and improving cheating detection. The proposals offered by the committee are:
As well, the committee recommends that the BoC publish more aggregate statistics specific to each department to better aid the faculty in recognizing trends related to dishonesty cases. The BoC could also more heavily publicize the general procedure for BoC cases and be more transparent about the outcome of cases to instructors.
At the conference, faculty members noted that the increase in the number of BoC cases may be a result of increased detection efforts rather than an increase in the prevalence of cheating. Faculty and students expressed support for classes providing a “collaboration table” that clearly indicates whether students can use particular resources when completing sets or exams; this policy has already been implemented by the mathematics department.
Students and faculty expressed concern that the revision of the 50-foot rule would increase the load on teaching assistants, potentially reducing the amount of support that students receive. Committee members noted that CS2, which implemented the revised policy for the 2018-19 school year, has been experimenting with strategies to deal with the problem, such as allowing certain designated students to view others’ code for the purposes of debugging.
[1] https://www.nytimes.com/2017/05/29/us/computer-science-cheating.html