Course Evaluations: Concerns with Gender and Racial Bias

Standardized student course evaluations play an important role in the review of faculty for merit raises, reappointment, tenure, and promotion.  While research recognizes course evaluations are an imperfect measure, the literature indicates many of the problems with course evaluations are unevenly distributed across men and women and across white faculty and faculty of color.

 

Social science research has documented gendered evaluation processes in a variety of institutional settings, including the academic workplace. These processes systematically put women at a disadvantage when their performance is being rated. Most recent exploratory studies of students’ evaluation of college and university teaching suggest that the same processes at work elsewhere in society are present when students evaluate professors.

 

For instance, one study (Sprague and Massoni, 2005) shows that when students describe good men teachers, they use many descriptions (e.g., funny) that are different from the ones they use to describe good women teachers (e.g., nurturing). The teaching activities that prompt these gendered descriptions differ. As a result, women may have to engage in more time-demanding forms of pedagogy in order to garner the same global teaching ratings as men who teach according to conventional gender expectations. In another study (MacNell et al., 2014) instructors in an online course presented themselves as both male and female to separate class sections, masking their true gender identities. The study concluded that “regardless of actual gender or performance, students rated the perceived female instructor significantly more harshly than the perceived male instructor.” There is also evidence that younger women are particularly disadvantaged, in comparison to younger men. Mengel et al (2017) used student evaluations from courses where students were randomly assigned to a male or female instructor and found that women, particularly junior women, received worse evaluations, even though instructor gender did not shape student study hours or grades.

 

Studies have also found that faculty of color received lower course evaluations than their white peers (Hamermesh & Parker, 2005; DiPietro & Faye, 2005). In a study of one university, DiPietro and Faye (2005) found that Hispanic faculty received the lowest course evaluation ratings. Asian-American faculty received slight better course evaluations than their Hispanic colleagues, but their scores were, on average, still worse than the scores of White faculty. Unfortunately, far less is known about evaluations of African American professors; this is largely due to the fact that there are so few African American professors in higher education. And, even less is known about the interactive effects of race and gender biases in teaching evaluations.

 

If you are interested in better understanding the concerns of bias with courses evaluations, we encourage you to review the archive of classic and recent research on this issue below.

Google Drive Folder of Archive of Research on Biases in Evaluations

Summaries of Articles of Research on Biases in Evaluations

You can submit an article or correction to the list here or by emailing Rebecca Kreitzer or contacting via twitter with any corrections or additions.

  Bibliography of Research on Gender and Racial Bias with Teaching and Evaluations

  1. Abel, M. H. and A. L. Meltzer, Students ratings of a male and female professors’ lecture on sex discrimination in the workforce, Sex Roles, 57, 173-180. (abstract)
  2. Anderson, Kristin J. Students’ stereotypes of professors: An exploration of the double violations of ethnicity and gender. Social Psychology of Education: AN International Journal. (2010) qe, 459-472 (link)
  3. Anderson, Kristin J. and M. Kanner. Inventing a gay agenda: Students’ perceptions of lesbian and gay professors. Journal of Applied Psychology, (2011) 41-1538-1564 (abstract)
  4. Anderson, Kristin J. and E. Miller. Gender and student evaluations of teaching. Political Science & Politics, 30(2):216-219, June 1997.
  5. Anderson, Kristin J. and Gabriel Smith. 2005. Students’ Preconceptions of Professors: Benefits and Barriers According to Ethnicity and Gender. Hispanic Journal of Behavioral Sciences
  6. Arbuckle, J. and B. D. Williams, Students’ perceptions of expressiveness: Age and gender effects on teacher evaluations, Sex Roles, 49 (9/10), 507-516. (abstract)
  7. Baldwin, T. and N. Blattner, Guarding against potential bias in student evaluations. What every faculty member needs to know, College Teaching, 51(1) (2003), 27-32.
  8. Baker, P & M Copp, Gender performance matters most: The interaction of gendered expectations, feminist course content and pregnancy in students' course evaluations. Teaching Sociology 25(1) (1997), 29-43.
  9. Basow, S. A. Best and worst professors: Gender patterns in students’ choices, Sex Roles, 43(⅚) (2000), 407-417.
  10. Basow, S. A. Gender dynamics in the classroom. In J. C. Chrisler, C. Golden, & P. D. Rozee, Lectures on the psychology of women (2nd ed.). New York: McGraw-Hill (2000).
  11. Basow, S. A. and N. T. Silberg, Student evaluation of college professors: Are female and male professors rated differently?, Journal of Educational Psychology 79 no. 3 308-314 (1987). (abstract)
  12. Basow, S. A. Student evaluations of college professors: When gender matters, Journal of Educational Psychology, 87, 656–665 (1995). (abstract)
  13. Basow, S. A. and  S. Montgomery, Student rating and professor self-ratings of college teaching: Effects of gender and divisional affiliation, Journal of Personnel Evaluation in Education, 18, 91-106, (2005).
  14. Bennett, S. K. Student perceptions of and expectations for male and female instructors: Evidence relating to the question of gender bias in teaching evaluations, Journal of Educational Psychology 74 no. 2 (1982), 170-179.
  15. Boring, A., K. Ottoboni, and P. B. Stark, Student Evaluations of Teaching (Mostly) Do Not Measure Teaching Effectiveness. ScienceOpen Research, 2016. (doi:10.14293/S2199-1006.1.SOR-EDU.AETBZC.v1)
  16. Boring, A. Gender Biases in Student Evaluations of Teachers and their Impact on Teacher Incentives, Working Paper, OFCE-PRESAGE-SCIENCES PO and LEDa-DIAL, April 22, 2015. (abstract)
  17. Boring, A., Gender biases in student evaluations of teaching, J. of Public Economics, 145 (January 2017), 27-41. (abstract)
  18. Bray, J. H. and G. S. Howard, Interaction of Teacher and Student Sex and Sex Role Orientations and Student Evaluations of College Instruction, Contemporary Educational Psychology 5 (1980), 241-248. (abstract)
  19. Brooks, V. R. Sex differences in student dominance behavior in female and male professors' classrooms. Sex Roles, 8 (7) (1982), 683-690.
  20. Buck, S. and D. Tiene, The combined effects of physical attractiveness, gender, and teaching philosophy on evaluations,The Journal of Educational Research, 82, no. 3, 172-177 (1989) (published online January 20, 2015, 172-177). (abstract)
  21. Burns-Glover, A. L. & D. J. Veith, Revisiting gender and teaching evaluations: Sex still makes a difference. Journal of Social Behavior and Personality, 10, 69–80 (1995).
  22. Chamberlain, M. S. and J. S. Hickey, Student evaluation of faculty performance: The role of gender expectations in differential evaluations, Educational Research Quarterly, 25(2) (2001) 3-14.
  23. DiPietro, M. & A. Faye. (2005). Online student-ratings-of-instruction (SRI) mechanisms for maximal feedback to instructors. Paper presented at the 30th Annual Meeting of the Professional and Organizational Development Network; Milwaukee, WI.
  24. Drake, S., A. Auletto, and J. Cowen. Grading Teachers: Race and Gender Differences in Low Evaluation Ratings and Teacher Employment Outcomes. American Educational Research Journal (2019).
  25. Dukes, R. and G. Victoria. The Effects of Gender, Status and Effective Teaching on the Evaluation of College Instruction. Teaching Sociology Vol. 17, No. 4 (Oct., 1989), pp. 447-457
  26. El Alayi, Amani, Ashley A. Hansen-Brown, Michelle Ceynar.  Dancing Backwards in High Heels: Female Professors Experience More Work Demands and Special Favor Requests, Particularly from Academically Entitled Students. Sex Roles 79(3-4), 136-150. (2018)
  27. Elmore, P. B. and K. A. LaPointe, Effects of Teacher Sex and Student Sex on Evaluation of College Instructors, Journal of Higher Education 66 no. 3 (1974).
  28. Elmore, P. B. and K. A. LaPointe, Effects of teacher sex, student sex & teacher warmth on evaluation of college instructor, Journal of Higher Ed Psychology 67 no. 3 (1975).
  29. Ewing, Vanessa Lynn, Arthur A. Stokes, and Eugene P. Sheehan. Prejudice against gay male and lesbian lecturers.The Journal of Social Psychology, 143(5), 569-578, 2003.
  30. Fan, Y., L.J. Shepherd, E. Slavich, D. Waters, M. Stone, R. Abel, and E.L. Johnston. “Gender and cultural bias in student evaluations: Why representation matters.” PLOS One 2019 https://doi.org/10.1371/journal.pone.0209749
  31. Ferber, M.A. and J. Huber. Sex of Student and Instructor: A Study of Student Bias. American Journal of Sociology, 80(4), 1975.
  32. Feldman, K. A. College students’ views of male and female college teachers: Evidence from the social laboratory and experiments – Part 1. Research in Higher Education, 33, 317–375 (1992).
  33. Feldman, K. A. College students’ views of male and female college teachers: Evidence from the social laboratory and experiments – Part 2. Research in Higher Education, 34, 151–211 (1993).
  34. Freeman, H. R. Student evaluations of college instructors: Effects of type of course taught, instructor gender and gender role, and student gender. Journal of Educational Psychology, 86(4), (1994)  627-630.
  35. Freiderike, M., J. Sauermann, & U. Zolitz, Gender bias in teaching evaluations, Institute of Labor Economics, September, (2017)
  36. Greenwald, A. G. & G. M. Gillmore, No pain, no gain? The importance of measuring course workload in student ratings of instruction, J of Educational Psychology, vol. 89, no. 4, 743-751 (1997).
  37. Hammermesh, Daniel and Amy Parker. Beauty in the classroom: instructors’ pulchritude and putative pedagogical productivity. Economics of Education Review 24 (2005) 369-376
  38. Heilman, M. E., and T. G. Okimoto, Why Are Women Penalized for Success at Make Tasks? The Implied Communality Deficit, Journal of Applied Psychology 92(1) (2007), 81-92.
  39. Hessler, M, D. M. Popper, D. Hollstein, H. Ohlenburg, P. H. Amemann, C. Massoth, L. M. Seidel, Z. Zarbock, W. Wenk. Availability of cookies during an academic course session affects evaluation of teaching. Med Edu 52(10: 1064-1072 (2018)
  40. Hogan, C. Review of the literature: The evaluation of teaching in higher education, Instructional Development Center, McMaster University (1978).
  41. Huston, T. (2005) "Race and Gender Bias in Higher Education: Could Faculty Course Evaluations Impede Further Progress Toward Parity?," Seattle Journal for Social Justice: Vol. 4 : Iss. 2 , Article 34.
  42. Kaschak, E. Another look at sex bias in students evaluations of professors: Do winners get the recognition that they have been given? Psychology of Women Quarterly, Summer, l981.
  43. Kaschak, E. Sex bias in students' evaluations of professors' teaching methods. Psychology of Women Quarterly, 3 (3) (1978), l35-l43.
  44. Kierstead, D., D'Agostino, P., & Dill, H. (1988). Sex role stereotyping of college professors: Bias in students' ratings of instructors. Journal of Educational Psychology, 80(3), 342-344.
  45. Laube, H., K. Massoni, J. Sprague, A. Ferber, The impact of gender on the evaluation of teaching: What we know and what we can do. NWSA Journal, 19(3), (2007) 87-104.
  46. Lombardo, J. P. and M. Tocci, Attribution of positive and negative characteristics of instructor as function of attractiveness and sex of instructor and sex of subject, Perceptual & Motor Skills, 48 (1979).
  47. MacNell, L., A. Driscoll, and A. N. Hunt, What’s in a name: Exposing Gender Bias in Student Ratings of Teaching, Innovative Higher Education, 40 (4), pp 291-303 (2015) (published online 05 December 2014). (abstract)
  48. McPherson, M. A., R. T. Jewell, & M. Kim, What determines student evaluation scores? A random effects analysis of undergraduate economics classes, Eastern Economic Journal, 35, 37-51 (2009).
  49. Martin, E. Power and Authority in the Classroom: Sexist Stereotypes in Teaching Evaluations. Journal of Women in Culture and Society, 9 (1984), 482-492.
  50. Martin, L. Gender, Teaching Evaluations, and Professional Success in Political Science. Presented at the American Political Science Association Annual Meeting, 2013. (abstract)
  51. Mengel, F., J. Sauermann, & Ulf Zölitz, Gender Bias in Teaching Evaluations, Journal of the European Economic Association, September 2017. (abstract)
  52. Miles, P. and D. House, The Tail Wagging the Dog;An Overdue Examination of Student Teaching Evaluations, International Journal of Higher Education, 40, no. 2, 2015. (abstract)
  53. Miller, J. & M. Chamberlin, Women are teachers, men are professors: A study of student perceptions, Teaching Sociology, 28 (4), 283– 298 (2000). (abstract)
  54. Mitchell, K. and J. Martin, Gender Bias in Student Evaluations Political Science & Politics, 2018, 51(3): 648-652. (abstract)
  55. Moore, M. Student resistance to course content: Reactions to the gender of the messenger. Teaching Sociology 25(2) (1997), 128-33.
  56. Peterson, D., L. Biederman, D. Anderson, T. Ditonto, K. Roe. 2019. Mitigating gender bias in student evaluations of teaching. PLOSOne.
  57. Piatak, Jaclyn and Zachary Mohr. 2019. More gender bias in academia? Examining the influence of gender and formalization on student worker rule following. Journal of Behavioral Public Administration. 2(2)
  58. Pittman, Chavella. “Race and Gender Oppression in the Classroom: The Experience of Women Faculty of Color with White Male Students.” Teaching Sociology, 2010.
  59. Punyanunt-Carter, N. and S. Carter. Students’ Gender Bias in Teaching Evaluations. Higher Learning Research Communications, 5(3), 28-37. http://dx.doi.org/10.18870/hlrc.v5i3.234
  60. Reid, Landon. D. 2010. The role of perceived race and gender in the evaluation of college teaching on RateMyProfessors.Com. Journal of Diversity in Higher Education, 3(3), 137-152.
  61. Rivera, L. and A. Tilcsik. “Scaling Down Inequality: Rating Scales, Gender Bias, and the Architecture of Evaluation.” January 2019. https://doi.org/10.31235/osf.io/j2tw9
  62. Rosen, A. S. Correlations, Trends, and Potential Biases among Publicly Accessible Web-Based Student Evaluations of Teaching:A large-scale study of RateMyProfessors.com data, Assessment & Evaluation in Higher Education January 2017: 1–14. (abstract)
  63. Sandler, B. R. Women faculty at work in the classroom, or why it still hurts to be a woman in labor. Communication Education (1991)(January), 6-15.
  64. Schuster, M. R. & S. R. Van Dyne, The changing classroom. In M. R. Schuster & S. R. Van Dyne (Eds.), Women's place in the academy, (pp. 161-171). Totowa, New Jersey: Rowman & Allanheld (1985).
  65. Sidanius, J. & M. Crane, Job evaluation and gender: The case of university faculty, Journal of Applied Social Psychology, 19(2) 174-197, (1989).
  66. Sinclair, L., Z. Kunda. Motivated stereotyping of women: She’s fine if she praised me but incompetent if she criticized me. Personality and Social Psychology Bulletin, 26(11), 1329-1342. http://dx.doi.org/10.1177/0146167200263002
  67. Smith, B. and B. Hawkins. Examining Student Evaluations of Black College Faculty: Does Race Matter?, The Journal of Negro Education, 2011, 80(2): 149-162.
  68. Spooren, P., B. Brockx, & D. Mortelmans, On the validity of student evaluation of teaching: The state of the art, Review of Educational Research, 83(4) 598-642, (2013).
  69. Sprague, J. & K. Massoni, Student evaluations and gendered expectations: What we can't count can hurt us. Sex Roles, 53, 779–793 (2005).
  70. Stark, P. B. and Richard Freishtat. An Evaluation of Course Evaluations, ScienceOpen, 26 September 2014.
  71. Statham, A., L. Richardson, and J. Cook, Gender and University Teaching: A Negotiated Difference, SUNY Press (1991).
  72. Storage, D., Z. Horne, A. Cimpian, & S-J. Leslie, The Frequency of “Brilliant” and “Genius” in Teaching Evaluations Predicts the Representation of Women and African Americans across Fields, PLOS ONE, March 3, 2016.
  73. Subtirelu, N. “She does have an accent but …” : Race and language ideology in students’ evaluations of mathematics instructors on RateMyProfessors.com. Language in Society February 2015, 44(1), 35-62.
  74. Uttl, B., C. A. White, and D. W. Gonzalez, Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related, Studies in Educational Evaluation, 54, September 2017, 22-42. (abstract)
  75. Wagner, N., M. Rieger, and K. Voorvelt, Gender, ethnicity and teaching evaluations: Evidence from mixed teaching teams, Economics of Education Review, vol. 54, 79-94 (October 2016). (abstract)
  76. Young, S., L. Rush, & D. Shaw, Evaluating gender bias in ratings of university instructors’ teaching effectiveness, International Journal of Scholarship of Teaching and Learning, 3, 1–14 (2009)

Bibliography of Research Showing Negligible or No Race or Gender Bias in Student Evaluations

  1. Hancock, G., D. Shannon, and L. Trentham. Student and Teacher Gender in Ratings of University Faculty: Results from Five Colleges of Study. Journal of Personnel Evaluation in Education 6:235-248, 1993.Wallisch, P. and J. Cachia. 2019. “Determinants of perceived teaching quality: The role of divergent interpretations of expectations.” 10.31234/osf.io/dsvgq
  2. Wallisch, P. and J. Cachia. 2019. “Determinants of perceived teaching quality: The role of divergent interpretations of expectations.” 10.31234/osf.io/dsvgq
  3. Wright, S. L. & M. A. Jenkins-Guarnieri, Student evaluations of teaching: Combining the meta-analyses and demonstrating further evidence for effective use, Assessment & Evaluation in Higher Education, 37(6) 683-699, (2012).

Bibliography of Research Showing Women Receive Higher Student Evaluations

  1. Bachen, C. M., M. M. McLoughlin, & S. S. Garcia, Assessing the role of gender in college students’ evaluations of faculty, Communication Education, 48, 193–210 (1999). (abstract)
  2. Centra, J. A. and N. B. Gaubatz, Is there gender bias in student evaluations of teaching? The Journal of Higher Education, 71(1), (2000) 17-33.
  3. Rowden, G. V. and R. E. Carlson, Gender issues and students' perceptions of instructors' immediacy and evaluation of teaching and course, Psychological Reports, 78, 835-839, (1996).
  4. Whitworth,  J. E., B. A. Price, & C. H. Randall, Factors that affect college of business student opinion of teaching and learning, Journal of Education for Business, May/June (2002), 282-289.
  5. Wigington, H., N. Tollefson, & E. Rodriguez, Students’ ratings of instructors revisited: Interactions among class and instructor variables. Research in Higher Education, 30(3), 331-34, (1989).

Reports, Recommendations and Newsletters

  1. ADVANCE-Nebraska, Annual Evaluation of Faculty Best Practices, March 2013.
  2. Basow, S. A. Student ratings of professors are not gender blind, Association for Women in Mathematics Newsletter, 24, no. 5 (Sept.-Oct. 1994).
  3. Basow, S. A. and J. L. Martin, Bias in student evaluations, in M.E. Kite (Ed), Effective Evaluation of teaching: A guide for faculty and administrators, (40-49), (2012).
  4. Benton, S. L. and W. E. Cashion, Student rating of teaching: A summary of research and literature (IDEA Paper #50), (2012).
  5. Gender Bias in Faculty Teaching Evaluations:Key Journal Articles and Additional Resources (Result of RIT’s NSF ADVANCE Grant; includes list of work of other NSF ADVANCE institutions with links)
  6. Fich, F. E. Are student evaluations of teaching fair?, Computing Research News 15(3): 2-10 (2003).
  7. Hensel, N. Realizing Gender Equality in Higher Education, ASHE-ERIC Higher Education Report 2, (1991).
  8. Lehigh (University) ADVANCE Men Advocates:Statement on Gender Bias in Student Course Evaluations
  9. Lilienfeld, E. How Student Evaluations are Skewed against Women and Minority Professors, Commentary:Higher Education, The Century Foundation, February 13, 2017.
  10. D. Savonick and C. Davidson, Gender Bias in Academe: An Annotated Bibliography of Important Recent Studies, HASTAC, January 26, 2015. (This page contains a wealth of information and is regularly updated)

New articles, blog posts and podcasts on bias in student evaluations

  1. L. Bates, Female academics face huge sexist bias – no wonder there are so few of them The Guardian, February 13, 2015.
  2. A. Boring, K. Ottoboni, nad P. Stark. Student evaluations of teaching are not only unreliable, they are significantly biased against female instructors. LSE Impact Blog, 2/4/2016
  3. K. Chow. Study: At ‘Rate My Professors,’ A Foreign Accent Can Hurt A Teacher’s Score. NPR, 3/5/2015.
  4. J. Cohen, L. Hinkson, and G. Rossman. Gender Bias in Teaching Evaluations. Annex Podcast, 4/11/2019.
  5. N. Collins, There’s No Easy Fix for Gender Bias in Student Evaluation of Teachers, Pacific Standard, January 8, 2016.
  6. A. Eidinger. She’s Hot: Female Sessional Instructors, Gender Bias, and Student Evaluations. Active History, 3/30/2017.
  7. M. Farr. “Arbitration decision on student evaluations of teaching applauded by faculty.” University Affairs. August 2018.
  8. C. Flaherty. Zero Correlation Between Evaluations and Learning. Chronicle of Higher Education, 9/21/2016.
  9. C. Flaherty. Teaching Evals: Bias and Tenure. Inside Higher Ed, 5/20/2019.
  10. T. Jacobs. How to Combat Gender Bias in Teaching Evaluations. Pacific Standard Magazine, 5/16/2019.
  11. A. Kamenetz. Why Female Professors Get Lower Ratings. NPR, 1/25/2016.
  12. E. Key, and P. Ardoin. Students rate male instructors more highly than female instructors. We tried to counter than hidden bias. Washington Post Monkey Cage blog. 8/20/2019.
  13. N. Koblitz, Are Student Ratings Unfair to Women, Association for Women in Mathematics Newsletter 20 no. 5 (1990), 17-19.
  14. N. Koblitz and R. Kleeberg, Bias and other factors in student ratings, Chronicle of Higher Education 93/9/1 (1993), B3.
  15. N. Kornell. Do the best professors get the worst ratings? Psychology Today, 5/31/2013.
  16. A. Marcotte, Best Way For Professors to Get Good Student Evaluations? Be Male, Slate, December 9, 2014.
  17. Ray, V. Is Gender Bias an Intended Feature of Teaching Evaluations, Inside Higher Ed, February 9, 2018. (abstract)
  18. L. Rivera and A. Tilcsik. One Way to Reduce Gender Bias in Performance Reviews. Harvard Business Review. 4.17.2019.
  19. R. Schuman. Needs Improvement: Student evaluations of professors aren’t just bad and absurd - they don’t even work. Slate. 4/24/2014
  20. E. Voeten. Student evaluations of teaching are probably biased. Does it matter? Monkey Cage Blog, Washington Post. 10/2/2013.

Bibliography of Other Ways Teaching Evaluations are Biased

  1. B. Uttle, and D. Smibert. Student evaluations of teaching: teaching quantitative courses can be hazardous to one’s career. Journal of Life and Environmental Sciences. (2019)https://doi.org/10.7717/peerj.3299

Tools to evaluate bias in student evaluations

  1. B. Schmidt. “Gendered Language in Teaching Evaluations” (interactive chart)

Suggested Citation: Holman, Mirya, Ellen Key and Rebecca Kreitzer. 2019. "Evidence of Bias in Standard Evaluations of Teaching."  We appreciate the assistance of undergraduate students and research assistants in maintaining this work.