Columbia Journal of Gender and law 731.1 A BETTER TENuRE BATTlE: FiGHTiNG BiAS iN TEACHiNG EVAluATiONS MEERA E. DEO* As legal education undergoes significant changes with regard to both student enrollment and faculty hiring, the fight to keep law faculty tenure is at the forefront. But the focus on tenure should be about the standards themselves. A better approach would be to change tenure requirements to create a more just and inclusive set of standards and criteria for evaluation. This Article draws from the first systematic, comprehensive, mixed-method empirical law faculty diversity study to investigate how challenges in the classroom and bias in teaching evaluations affect female law faculty of color. The in-depth interviews of female law faculty of color are systematically analyzed using Atlas.ti software, finding that students directly challenge particular faculty in class, sometimes through verbal and even physical abuse, and write insensitive and irrelevant race- and gender-based comments on anonymous teaching evaluations. These encounters often have negative effects on the professional trajectory of women of color law professors, most notably when these individuals seek promotion and tenure. Instead of supporting these discriminatory barriers to advancement, legal institutions should do away with student evaluations altogether, modify them, or supplement them with more rigorous and less discriminatory forms of evaluation. This is the way to fight bias in teaching evaluations. TABlE OF CONTENTS introduction.………………………………………………….....8 i. The lay of the land………………………………….……….11 A. Statistics on law Faculty Members…………………….11 B. law Faculty Experiences………………………………..13 C. Biased Course Evaluations.………………………………14 ii. The Diversity in legal Academia Project.…………………17 * Associate Professor of Law, Thomas Jefferson School of Law; J.D., Ph.D. The author, who was an organizer of the Crosscutting Program on Persisting Inequalities in Legal Academia presented at the 2015 AALS Annual Meeting, thanks the AALS Section on Women in Legal Education and the twelve additional AALS Sections that supported this topic. Helpful research assistance was provided by Kale Sopoaga, Brittney Nobles, and Jillian Kates. This Article and the Diversity in Legal Academia (“DLA”) project as a whole could not be possible without the enthusiastic participation of the ninety-seven legal academics who contributed their experiences and their voices to the study. Columbia Journal of Gender and law8 31.1 iii. Challenging Confrontations and Biased Comments………22 A. Classroom Confrontations.……………………………22 1. Disappointing Students’ Expectations.……………22 2. Examples of Confrontations………………………..24 B. Biased Evaluations…………………………………….…25    1.  Channeling Sofia Vergara…………………………….26 2. Microaggressions and Cruel Comments……………27 C. Contrasts..………………………………………………31 1. Other “Outsider” Experiences……………………….31 2. White Male Experiences………………………………33 iV. Responses to Student Bias……………………………………34 A. individual Strategies……………………………………35 1. Avoidance…………………………….………………36 2. Commanding Authority……………………………36 B. Structural Solutions……………………………………39 Conclusion…………………………………………………….…41 iNTRODuCTiON Legal education is changing, shrinking, adapting, and evolving. The past few years have seen dramatic declines in the number of students taking the Law School Admissions Test (generally a requirement for law school admission), with rates of test-takers dropping forty-five percent since October 2009 and currently at the lowest rate since 1998.1 One unsurprising result is that law school admissions and enrollment have dropped significantly as well.2 With fewer students enrolling, law schools themselves are responding by shrinking their faculties, encouraging some to retire, and being less willing to hire new tenure-track candidates.3 1 The ABA Journal and numerous news outlets have highlighted declining student admissions and enrollment, foreshadowed by the drop in LSAT takers. See Debra Cassens Weiss, LSAT Test-Takers Continue to Decline; October Number is at Lowest Point Since 1998, a.b.a. J. (Nov. 4, 2013), http://www.abajournal.com/ news/article/lsat_test_takers_continue_to_decline_october_number_is_at_lowest_point_sinc [http://perma.cc/ 94DV-K97F] (last visited Jan. 12, 2014). 2 See brian Z. Tamanaha, FailinG law SChoolS (2012); Luz E. Herrera, Educating Main Street Lawyers, 63 J. leGal eduC. 189 (2013); Philip G. Shrag, Failing Law Schools—Tamanaha’s Misguided Missile, 26 Geo J. leGal eThiCS 387 (2013). 3 See, e.g., Ashby Jones & Jennifer Smith, Amid Falling Enrollment, Law Schools Are Cutting Faculty, wall STreeT J. (July 15, 2013), http://www.wsj.com/articles/SB100014241278873236642045786078102924 33272 [http://perma.cc/4MWG-RY3T?type=live]. Columbia Journal of Gender and law 931.1 In addition to these direct market-driven changes, regulations and standards for legal education are shifting. Many state standards are changing to reflect a desire from many prospective employers for new law graduates to be more highly trained and effective from the start of their careers, rather than relying on firms and organizations to train and nurture new lawyers into effective advocates. For instance, California has moved to require additional skills-based training in law school, including “15 units of practice-based, experiential course work or an apprenticeship equivalent” and “50 hours of legal services devoted to pro bono or modest means clients” within one year of graduation.4 The American Bar Association has also considered whether it should revise its long- standing requirement that member schools offer faculty tenure.5 Perhaps unsurprisingly, this last provision has seen especially vocal and passionate opposition by law faculty.6 The arguments that law schools should retain the requirement of tenure seem more focused on equal opportunity, academic freedom, and the fear that allowing schools to remove tenure will lead to less protection for the most marginalized individuals—especially those who voice politically-unpopular opinions.7 There have also been a few academic proponents of the move to abolish tenure.8 4 The California Bar Journal has been carefully following these developments. See, e.g., Laura Ernde, Panel Outlines Plans to Amend Rules for Attorney Training, Cal. b. J., http://calbarjournal.com/January2014/ TopHeadlines/TH4.aspx [http://perma.cc/54BZ-JBP8] (last visited Jan. 13, 2014). 5 Paul Caron, ABA Approves Dilution of Faculty Tenure, Employment Data Reporting Change Requested by CA, NY Law Schools, Tax ProF bloG (Aug. 10, 2013), http://taxprof.typepad.com/taxprof_blog/2013/08/ aba-approves-.html [http://perma.cc/T6CX-WHP4]. 6 See, e.g., Society of American Law Teachers (“SALT”), Statement on Tenure/Security of Position, (Apr. 2, 2011), http://www.americanbar.org/content/dam/aba/migrated/2011_build/legal_education/ committees/standards_review_documents/20110328_comment_security_of_position_salt.authcheckdam.pdf [http://perma.cc/EY7Z-JXFW]; Deborah Post, The Attack on Tenure, SalT (Aug. 10, 2010), https://www. saltlaw.org/the-attack-on-tenure-downwardly-mobile-professionals [https://perma.cc/CU64-RAPX]; Ruben Garcia, Tenure Standard 405 Remains, SalT (Mar. 19, 2014), https://www.saltlaw.org/tenure-standard-405- remains [https://perma.cc/TMH6-5GMQ] (“The Society of American Law Teachers (SALT) has consistently advocated in favor of tenure and security of position in several comments submitted to the Council’s Standards Review Committee.”). 7 Letter from law professors to the Honorable Solomon Oliver, Jr., Council Chairperson, and Barry A. Currier, Managing Director of Accredation and Legal Education at the Section on Legal Education and Admissions to the Bar (Feb. 2014), http://blurblawg.typepad.com/files/law-professors-letter-to-the-aba-on- tenure.pdf [http://perma.cc/75SK-9PW6]. 8 One popular article is University of New Mexico School of Law Assistant Professor Dawinder “Dave” Sidhu’s op-ed, which ran in USA Today on August 25, 2013. Dawinder Sidhu, Get Rid of Tenure for Law Schools: Column, uSa Today (Aug. 25, 2013), http://www.usatoday.com/story/opinion/2013/08/25/law- Columbia Journal of Gender and law10 31.1 The purpose of this Article is not to voice support for or opposition to efforts to abolish tenure; others have argued both positions eloquently and forcefully. Rather, it is to recognize that while changes in legal education may pose challenges for legal institutions, they also create an opening to make positive change. While tenure is being called into question, we can take this opportunity to evaluate whether the tenure standards currently in place are the most fair, objective, and worthwhile measures. Law schools overwhelmingly evaluate tenure files based on the scholarship, teaching, and service of the applicants.9 This Article focuses on the second prong and specifically on ways to improve how we evaluate effective teaching. Law schools should place less reliance on teaching evaluations, which prior studies show have the potential to be biased against those underrepresented in legal academia—particularly women of color—while ironically not being directly related to teaching efficacy.10 In fact, my own mixed-method study of law faculty diversity, the Diversity in Legal Academia (“DLA”) project, indicates that female law faculty and particularly female faculty of color are routinely challenged by students in the classroom and verbally attacked in (anonymous) course evaluations.11 The findings discussed in this Article show that when evaluating these particular law faculty members, student attention is often focused on the professor’s personal style over the substance of her teaching, and on the race and gender of the professor rather than what or how she teaches. As a result, these underrepresented law faculty face unnecessary hurdles in the tenure process in spite of being effective teachers. Instead of relying on flawed student evaluations, peer teaching evaluations—from faculty at the applicant’s institution or even from outside local or national schools—can better determine whether faculty are prepared in the classroom and meeting the needs of the law school to prepare students for legal practice. Additionally, law schools should modify existing evaluations or eliminate them altogether, at least from the tenure process. school-tuition-column/2678487 [http://perma.cc/8YW6-KEP7]. 9 Most require that candidates display “excellence” in all three categories in order to advance to tenure. See generally Workshop for Pretenured People of Color Law School Teachers, aSS’n am. l. SCh., https://memberaccess.aals.org/eweb/DynamicPage.aspx?WebKey=C47336F7-15FC-4823-A248-8510D5 F2F917&Reg_evt_key=7a607a59-17aa-48bf-add3-7cd3594f280f [https://perma.cc/26EZ-5ZVT?type=image] (last visited Aug. 6, 2015). 10 See infra at Part I.C. 11 An earlier article drawing on data from the DLA project begins these investigations. See Meera E. Deo, The Ugly Truth about Legal Academia, 80 brook. l. rev. 943 (2015) [hereinafter Deo, Ugly Truth]. Further elaboration on both classroom confrontations and biased teaching evaluations are the focus of this Article. Columbia Journal of Gender and law 1131.1 Part I of this Article provides background on legal education and legal academics. It also covers some existing scholarship on bias in teaching evaluations. In Part II, the Article introduces the Diversity in Legal Academia (“DLA”) project, the first comprehensive, mixed-method empirical study of law faculty members from entry to the profession through retirement, including men and women from diverse racial/ethnic backgrounds, and with a focus on women of color law professors. Part III presents findings from DLA data showing how ongoing student confrontations of female faculty of color in the classroom set the stage for biased and irrelevant teaching evaluations. In Part IV, the Article draws from DLA data to suggest individual strategies and structural solutions for combatting the identified challenges. The Article concludes with three specific proposals for improving how schools measure excellence in teaching, in addition to suggesting recommendations for institutions committed to providing a fairer environment for all law professors. i. The lay of the land Numerous articles have celebrated the increasing numbers of women in law, including those highlighting the fact that women have been in the majority of many entering law school classes in recent years.12 However, women remain significantly underrepresented in legal academia, in spite of the burgeoning pool of potential female law faculty.13 In addition, scholarship has documented ongoing challenges facing female law faculty members, ranging from confrontational interactions with colleagues and students to actual bias in teaching evaluations.14 A. Statistics on law Faculty Members Women are underrepresented in legal academia even today, with women of color represented at especially low rates. AALS statistics for 2008–2009, the most recent year for which statistics are available, indicate that women from all racial/ethnic backgrounds 12 See, e.g., Joe G. Baker, The Influx of Women into the Legal Profession: An Economic Analysis, monThly lab. rev., Aug. 2002, at 14, http://www.bls.gov/opub/mlr/2002/08/art2full.pdf [http://perma.cc/AJ8X-YK46] (“The year 2001 was a watershed year in legal education. For the first time, female law school entrants outnumbered men.”). 13 Meera E. Deo, Looking Forward to Diversity in Legal Academia, 29 berkeley J. Gender l. & JuST. 352, 359–64 (2014) [hereinafter Deo, Looking Forward]. 14 For more on racial/gender differences with regard to interactions among law faculty colleagues and between law professors and law students, see Deo, Ugly Truth, supra note 11, at 943. Columbia Journal of Gender and law12 31.1 comprise just 37.3% of all legal academics. (See Table 1.)15 White men are the numeric majority in legal academia; in fact, there are 1,000 more white men in legal academia than there are women from all race/ethnic backgrounds (5,090 white men as compared to 4,091 women total). Of the 4,091 women in legal academia, the overwhelming majority (at least 67%) are white.16 When we consider the statistics for the 772 women of color law professors, African Americans make up the largest constituency (10%) while Native Americans are only .5% of all legal academics, with only 21 individuals in the legal academy. Table 1 legal Academics, by Race and Gender, AAlS 2008 (N=10,965) Race Ethnicity American indian or Alaskan Native Asian or Pacific  islander Black/ African American Hispanic/ latino White Other Race More than one race Race/ Ethnicity not identified Total Men N % 30 .4 158 2.3 344 5.0 199 2.9 5090 74.6 67 1.0 62 .9 869 12.7 6819 100.0 Women N % 21 .5 112 2.7 409 10.0 138 3.4 2741 67.0 34 .8 58 1.4 578 14.1 4091 100.0 Gender Not Identified N % 0 .0 0 .0 0 .0 0 .0 0 .0 0 .0 0 .0 55 100.0 55 100.0 Total N % 51 .5 270 2.5 753 6.9 337 3.1 7831 71.4 101 .9 120 1.1 1502 13.7 10965 100.0 15 While more recent statistics would be especially helpful, especially in discussing evolving legal changes, AALS no longer makes this data publicly available. The table remains on file with the author. 16 An additional 869 men and 578 women do not disclose their race/ethnicity. Social science literature suggests that those who do not identify their race/ethnicity tend to be white. Therefore, this 67% figure is likely underinclusive, with the actual numbers and percentages of whites in legal academia being significantly higher. Columbia Journal of Gender and law 1331.1 Table 2 Women of Color law Faculty, by Race, AAlS 2008-2009 (n=772) n % American Indian or Alaskan Native 21 2.7 Asian or Pacific Islander 112 14.5 Black/African American 409 53.0 Hispanic/Latino 138 17.9 Other Race 34 4.4 More than one race 58 7.5 Total 772 100 B. law Faculty Experiences In spite of—or perhaps because of—their numeric minority, women of color law faculty are often the target of challenges to classroom authority and venomous student evaluations. Even thirty years ago, the first symposium gathering law faculty of color documented classroom “challenges from white students who have come ‘[f]ace to face for possibly the first time in their lives with a minority person in a position of authority.’”17 A few years later, law professors Derrick Bell and Richard Delgado conducted a non-representative survey of law faculty, published in 1989 in the Harvard Civil Rights-Civil Liberties Law Review.18 The Bell-Delgado study found that the majority of law professors of color enjoyed at least “satisfactory” relationships with white students.19 Yet a significant number reported direct challenges by white students specifically.20 For instance, white students were comfortable complaining to their Black professors that they were disappointed in how the class was taught.21 One group of students took it upon themselves to educate their Latino professor 17 Deo, Looking Forward, supra note 13, at 369 (quoting Roy L. Brooks, Life After Tenure: Can Minority Law Professors Avoid the Clyde Ferguson Syndrome?, 20 u.S.F. l. rev. 419, 420 (1986)). 18 Richard Delgado & Derrick Bell, Minority Law Professors’ Lives: The Bell-Delgado Survey, 24 harv. C.r. & C.l. l. rev. 349 (1989). 19 “Most minority professors reported satisfactory or good relationships with white students.” Id. at 359. Note that their study did not disaggregate by gender so it is unclear if female faculty of color enjoyed similarly positive interactions as their male colleagues of color. 20 Id. 21 Id. Columbia Journal of Gender and law14 31.1 on how he should be teaching the course, offering contradictory advice at that.22 One Black female faculty member from that study was frequently mistaken on campus for a maid, with “spills and messes [called] to her attention” with the expectation that she would clean them up.23 Recent scholarship suggests little improvement in the intervening quarter-century. A 2012 anthology of experiences, observations, and research entitled Presumed Incompetent documents the experiences of women of color in academia across the disciplines, with a primary theme being the presumption of incompetence that greets most women of color faculty at the door even before they begin teaching.24 The few chapters from that book centering on the law faculty experience continue the conversation that the Bell-Delgado study began years before. For instance, one Black female law professor notes that some of her students approached her soon after she began teaching to suggest ways that she could make them more comfortable in the classroom; their primary “advice” was that if she “came into class and gave them a big, warm smile every morning and continued smiling throughout the class, then perhaps they could accept me—a black female teacher— better.”25 Another Black woman recognized quickly how students responded to her as compared to their white professors, noting “students judged me more harshly than they did my white peers.”26 A Latina law professor had such a negative professional response, not only from students but from the administration, at a time of great personal challenges, that she “resigned from my tenure-track position” altogether rather than continue to endure the harsh treatment.27 C. Biased Course Evaluations Other scholarship has documented the ways in which students directly challenge the 22 Id. at 360. 23 Id. 24 See PreSumed inComPeTenT: The inTerSeCTionS oF raCe and ClaSS For women in aCademia (Gabriella Gutiérrez y Muhs, Yolanda Flores-Niemann, Carmen G. González & Angela P. Harris eds., 2012). 25 Angela Mae Kupenda, Facing Down the Spooks, in PreSumed inComPeTenT, supra note 24, at 20, 22. 26 Angela Onwuachi-Willig, Silence of the Lambs, in PreSumed inComPeTenT, supra note 24, at 142, 149. 27 Elvia R. Arriola, “No hay mal que por bien no venga”: A Journey to Healing as a Latina, Lesbian Law Professor, in PreSumed inComPeTenT, supra note 24, at 372, 372. Columbia Journal of Gender and law 1531.1 authority of women of color law professors in class.28 While some students may avoid the risk of a personal confrontation with professors, course evaluations provide all students with an opportunity to express their biases and preferences in an anonymous, yet powerful format. As others have noted, there is already “an avalanche of articles critiquing and defending student evaluations.”29 Many agree that teaching evaluations favor white male faculty over others, and also traditional faculty over anyone with an “outsider” status (i.e., women, people of color, those who speak with an accent, those perceived to be immigrants, foreigners, or people from a lower socioeconomic status, etc.).30 Yet, few studies rely on empirical data to support their arguments and few of these, if any, focus on legal academia. This Article is especially relevant at this particular time in legal academia, when changes are already underfoot and when faculty diversity may be slipping even further. Research has shown how comments on these evaluation forms often focus on the teaching, or even the personal style of the female professors, rather than the substance of the material taught. For instance, one particular study utilized a charming actor to spout gibberish rather than teach the actual material to a class of undergraduate students; the students rewarded him with higher scores than a more forthright expert in the field who taught actual substance to her class.31 What some have termed the “Miss Congeniality” bias means that highly expressive, friendly, witty, and entertaining instructors receive higher scores than others, regardless of their ability to master or convey the substantive material of the course.32 Earlier studies showed what more recent studies confirm, that “‘expressiveness’ in the case of students who were not highly motivated” (in this case undergraduate general education courses, but similar to how many would characterize most of the traditional first-year law school courses) trumps on student evaluations even 28 Deo, Ugly Truth, supra note 11, at 993–99. 29 Sylvia R. Lazos, Are Student Teaching Evaluations Holding Back Women and Minorities?: The Perils of “Doing” Gender and Race in the Classroom, in PreSumed inComPeTenT, supra note 24, at 164, 165. 30 See, e.g., William Arthur Wines & Terence J. Lau, Observations on the Folly of Using Student Evaluations of College Teaching for Faculty Evaluation, Pay, and Retention Decisions and its Implications for Academic Freedom, 13 wm. & mary J. women & l. 167, 175 (“Minority faculty members receive lower teaching evaluations than do majority professors, and non-native English speakers receive substantially lower ratings than do natives.”). 31 Lazos, supra note 29, at 168–69. 32 Wines & Lau, supra note 30, at 177–78. Columbia Journal of Gender and law16 31.1 “the amount of content covered in the course.”33 In other words, students reward smiles more than content. While faculty generally believe that excellence in teaching requires fairness in grading and the appropriate pedagogical approaches, students instead tend to emphasize whether a professor “smiles and displays a friendly demeanor, treats students as equals, greets students encountered outside of class, [and] adjusts course material to fit student interests.”34 Studies have also documented that students award higher evaluations to women of color whom they find more attractive—though no studies have shown that beauty correlates with teaching effectiveness.35 While the content of the course seems relatively unimportant, the distribution of chocolate while students complete course evaluations is directly correlated with an increase in objective measures of teaching effectiveness.36 One Black scholar who participated in a recent study recalls a student evaluation of her first-year law course stating that the student was more interested in the professor’s personal style than the substantive material she taught.37 In this way, women are rated not as professors per se, but on “a scale that applie[s] to women only—a scale that focuse[s] selectively on a woman’s style of dress, appearance, body, social graces, and other traits not directly linked to [her] ability to perform [her] job.”38 Thus, women of color law professors who seek to attain excellent reviews from students may have to think about what to teach, how to teach it, and even what to wear in front of their students; in these and other ways, “being a token is exhausting and creates extra work for the ‘outsider’ law professor.”39 There may also be inherent flaws in attempting to measure teaching effectiveness through quantitative measures on student surveys. Researchers who conducted a statistical analysis of teaching evaluations concluded that they “don’t measure teaching effectiveness. [They] 33 Wines & Lau, supra note 30, at 177. 34 Wines & Lau, supra note 30, at 178. 35 Lazos, supra note 29, at 167; see also Wines & Lau, supra note 30, at 175 (describing Beauty Bias). 36 Lazos, supra note 29, at 170. 37 Adrien Katherine Wing, Lessons from a Portrait: Keep Calm and Carry On, in PreSumed inComPeTenT, supra note 24, at 356, 356. 38 Martha Chamallas, The Shadow of Professor Kingsfield: Contemporary Dilemmas Facing Women Law Professors, 11 wm. & mary J. women & l. 195, 195 (2005). 39 Id. at 196 (citing Devon W. Carbado & Mitu Gulati, Working Identity, 85 Cornell l. rev. 1259, 1262 (2000)). Columbia Journal of Gender and law 1731.1 measure what students say, and pretend it’s the same thing.”40 For instance, because there is no objective determination of what a score of “5” on a survey means, averaging together one student awarding a teacher a “7” with another score of “3” does not automatically mean the teacher is performing at an average level of “5.”41 In addition, a number of variables cannot be accounted for in teaching evaluations yet are likely to skew results; these include the timing of the class (early morning classes yield notoriously low evaluations), regular attendance of students (posting slides may discourage students from actually attending), and class size (small sample sizes create unreliable results, while especially large class sizes lead to less individual attention per student).42 Ironically, even extremely consistent results may indicate flaws in student evaluations. Because it is highly unlikely that one professor is “equally effective with students with different background, preparation, skill, disposition, maturity, and ‘learning style,’” a professor earning very consistent ratings from all students is likely scoring points for something other than teaching effectiveness.43 Data from the DLA study indicate that women of color in legal academia confront ongoing challenges in the classroom and often vitriolic and biased teaching evaluations. Nevertheless, classroom success measured in part by these biased and flawed student evaluations is used routinely for tenure and promotion purposes, with poor marks on student evaluations often cited for advancement and tenure denials for women of color faculty.44 Rather than rely on the evaluations as they are, law schools should eliminate them altogether, change the evaluations to better measure particular markers of teaching success, and/or incorporate peer measures of teaching effectiveness as we consider revisions to tenure overall. ii. The Diversity in legal Academia Project The DLA project is the first formal empirical study of law faculty that investigates the personal and professional experiences of legal academics at all levels of the professional spectrum, from Assistant Professor to Dean Emeritus. This landmark study investigates 40 Philip B. Stark & Richard Freishtat, An Evaluation of Course Evaluations, SCienCeoPen 9 (Sept. 26, 2014), https://www.stat.berkeley.edu/~stark/Preprints/evaluations14.pdf [https://perma.cc/WHS3-M2P9]. 41 Id. at 6 (“They presume that a 3 ‘balances’ a 7 to make two 5s. For teaching evaluations, there’s no reason any of those things should be true (See, e.g., McCullough & Radson, 2011).”). 42 Id. at 4–5. 43 Id. at 12. 44 Lazos, supra note 29, at 185. Columbia Journal of Gender and law18 31.1 the law faculty experience through a target sampling technique utilizing surveys and in- depth interviews.45 The study includes participation from sixty-three women of color legal academics as the core sample, and thirty white men, white women, and men of color legal academics who serve as the comparison sample. Though most academics recognize race as a social construct (rather than a biological manifestation), defining or categorizing race and ethnicity remains a challenging endeavor.46 DLA participants self-identified their race, ethnicity, and gender on the survey instrument as well as during subsequent in-person interviews with me, the Principal Investigator of the DLA study.47 These self-identifications are used in the study to identify the race/ethnicity of each participent. The focus of the study is on women of color, meaning non-white women, and including women who identify as Black, Latina, Asian/Pacific Islander, Native American, Middle Eastern, and multiracial.48 Women law professors from these backgrounds make up the core sample, while white men, white women, and men of color from each of these racial/ethnic backgrounds constitute the comparison sample. All DLA participants are tenured or tenure-track faculty members at the time of data collection, whose home institutions at the time were ABA-accredited and AALS-member schools.49 Librarians, clinicians, and legal writing professors are excluded from the study, though some of these faculty members are tenured/tenure-track; because their experiences 45 For more on the methods, analytical approach, and initial hypotheses of DLA, see Deo, Looking Forward, supra note 13. 46 See, e.g., Ian F. Haney-Lopez, The Social Construction of Race: Some Observations on Illusion, Fabrication, and Choice, 29 harv. C.r. & C.l. l. rev. 1 (1994). 47 This Article uses the terms “African American” and “Black” interchangeably to refer to those who characterized themselves in the study using those terms. The terms “Latino” and “API” are used to refer to those who self-identified as Hispanic/Latino and Asian/Pacific Islander, respectively. The term “whites” refers to participants who identify as white (non-Hispanic) in the study. 48 Because these are the categories employed by AALS in their statistics (see supra note 15), they are used in the DLA study as well. 49 As Professor and former Berkeley Law Dean Herma Hill Kay explains, “The ABA and the AALS have different, but somewhat overlapping, standards for giving approval to law schools (in the case of the ABA) and for electing schools to membership in the AALS. The former is concerned with ensuring competence to train law students and uses more quantitative measures, while the latter is more concerned with faculty scholarship and uses more qualitative measures . . . . Combining these entrance qualifications for my study while omitting non-ABA accredited schools thus ensures that women professors at ABA-AALS schools have met the most rigorous standards used in legal academia.” Private correspondence with Dean Herma Hill Kay (Jan. 7, 2013) (on file with author). Columbia Journal of Gender and law 1931.1 tend to differ greatly from other faculty, most research on law faculty is specific to one group or the other.50 Table 3 Diversity in legal Academia Participants, by Race and Gender, DlA 2013 (n=93) Male Female Total Black 4 (4.3%) 21 (22.6%) 25 (26.9%) Asian/Pacific Islander 3 (3.2%) 15 (16.1%) 18 (19.4%) latino 2 (2.2%) 13 (14.0%) 15 (16.1%) Native American 1 (1.0%) 5 (5.4%) 6 (6.5%) Middle Eastern 1 (1.0%) 2 (2.2%) 3 (4.4%) Multiracial 1 (1.0%) 7 (7.5%) 8 (8.6%) White 7 (7.5%) 11 (11.8%) 18 (19.4%) Total 19 74 93 DLA participants completed an online survey that covered background personal and professional information, including date of birth, debt from law school, date of first 50 This study echoes the reasoning of Herma Hill Kay, who writes the following in her article UC’s Women Law Faculty: My definition of a “professor” includes only tenure or tenure-track assistant, associate, and full professors. It excludes librarians, clinicians, adjunct professors, and legal writing teachers, even though some of the women who were law librarians during this period held professorial appointments. . . . Others have drawn similar distinctions. See Marina Angel, Women in Legal Education: What It’s Like to be Part of a Perpetual First Wave Or the Case of the Disappearing Women, 61 TemP. l. rev. 799, 803 (1988) (including “only those teachers with visibility and power within their school; namely, tenured or tenure-track regular assistant professors, associate professors, or professors”); Deborah Jones Merritt & Barbara F. Reskin, Sex, Race, and Credentials: The Truth about Affirmative Action in Law School Hiring, 97 Colum. l. rev. 199, 206 (1997) . . . (stating that “we chose tenure-track law professors as the subject of our study because those faculty members hold influential posts, shaping both the next generation of lawyers and the development of legal doctrine.”); Elyce H. Zenoff & Kathryn V. Lorio, What We Know, What We Think We Know, And What We Don’t Know About Women Law Professors, 25 ariZ. l. rev. 869, 871–72 (1984) (counting only tenure-track faculty, defined as “professor, associate professor, or assistant professor, unmodified by any other term such as adjunct, clinical, visiting, or emeritus” and noting that “librarians, although usually tenure-track, were excluded because they constitute a distinct career line.”). Herma Hill Kay, UC’s Women Law Faculty, 36 u.C. daviS l. rev. 331, 337–38 n.27 (2003). Certainly, a distinct study focusing on these populations would add a layer of understanding to our conceptions of legal education. Columbia Journal of Gender and law20 31.1 tenure-track employment, and professional positions/titles.51 The survey also collected responses to experiential and attitudinal questions ranging from satisfaction in the profession to comfort in the current campus climate. Participants also submitted their CVs for content analysis.52 Once they completed the surveys, research subjects then participated in in-depth, one-on-one interviews that I conducted myself, either in person or telephonically. The interview protocol covered entry into the profession, mentor relationships, interactions with colleagues and students, work/life balance, sources of support, and suggestions for improving faculty diversity, among other topics. The mixed-method approach allows for full triangulation of the data so that all salient themes can be explored from various angles and perspectives.53 As the Principal Investigator of the project, I retain full responsibility for all aspects of DLA, from study design to implementation and dissemination. Data collection followed a target sample approach, often used when studying vulnerable or hidden populations because the social connections forged with initial participants encourages subsequent potential participants to join.54 In this case, a seed group of representative faculty members was carefully selected at the outset;55 this group then nominated other potential participants who were selected to participate based on a desire to ensure ongoing representativeness of the pool as a whole.56 Thus, the study began with participants who represented the full range of existing law faculty with regard to a number of domains, including: race/ethnicity, gender, region of the United States, selectivity 51 The survey was facilitated by the online research tool Survey Monkey. See Survey monkey, http://www. surveymonkey.com [http://perma.cc/PF5K-FYEY] for more information on this standard data collection tool. 52 “Content analysis is a widely used qualitative research technique . . . used to interpret meaning from the content of text data,” in this case, from the CVs of study participants. Hsiu-Fang Hsieh & Sarah E. Shannon, Three Approaches to Qualitative Content Analysis, 15 QualiTaTive healTh reS. 1277, 1277 (2005). 53 John w. CreSwell & viCki l. Plano Clark, deSiGninG and ConduCTinG mixed meThodS reSearCh 1 (2007); abbaS TaShakkori & CharleS b. Teddlie, mixed meThodoloGy: CombininG QualiTaTive and QuanTiTaTive aPProaCheS 5 (1998). 54 Katherine Browne, Snowball Sampling: Using Social Networks to Research Non-Heterosexual Women, 8 inT’l J. SoC. reS. meThodoloGy 47 (2005). 55 The original, or “seed,” participants were purposefully selected to be diverse with regard to various domains; each domain was then formally tracked whenever selecting additional participants from among those nominated. In addition, age, sexual orientation, and disability status were loosely tracked to ensure representation in the sample. 56 Browne, Snowball Sampling, supra note 54, at 47. Columbia Journal of Gender and law 2131.1 of institution,57 tenure status, professorial title (including leadership roles), and more.58 Many of the interviews for the DLA study occurred at national or regional conferences throughout the United States, to facilitate the participation of faculty from around the country. In addition, a number of interviews were conducted telephonically, to fully ensure representativeness and to capture perspectives from individuals who were not attending conferences over the 2013–2014 calendar year—including those on maternity leave, on sabbatical, or without the financial resources to travel. Each participant was assigned a pseudonym for data analysis and publication purposes in order to protect anonymity. These pseudonyms, rather than the actual names of participants, are presented in the findings sections. All interviews were transcribed and checked for clarity and errors. Coding and analysis were facilitated using ATLAS.ti software.59 A comprehensive codebook was developed using the specific questions asked on the protocol as well as separate themes that arose from preliminary analyses of the data. Descriptive content analyses of the interview transcripts yielded general themes and interpretations of findings. These were then coordinated with quantitative analyses of the survey data using Stata and Excel.60 Generally, the quantitative analyses are used to frame and contextualize the more in-depth qualitative findings presented in this Article. The analytical approach evaluates salient links between characteristics, i.e., race, ethnicity, gender, social class, region, and diversity of attitudes, while considering the contextual attributes and topics of each interview.61 This study received a formal regulatory opinion of IRB exemption from Western IRB.62 57 This was measured by U.S. News and World Report statistics. See Best Law Schools, u.S. newS & world reP. (2015), http://grad-schools.usnews.rankingsandreviews.com/best-graduate-schools/top-law- schools/law-rankings [http://perma.cc/P8GE-CZLL]. 58 For more on DLA methods, see Deo, Looking Forward, supra note 13. 59 See aTlaS.Ti, http://www.atlasti.com/index.html (last visited May 1, 2014); see also ann lewinS & ChriSTina Silver, uSinG SoFTware in QualiTaTive reSearCh (2007) (explaining the functions of the major software programs that are generally used in studies of this kind, including ATLAS.ti). 60 See generally STaTa, http://www.stata.com [http://perma.cc/HC3T-YDP2] (last visited July 29, 2015). Stata is a statistical software package developed and sold by StataCorp) (last visited Apr. 22, 2014); see also alan C. aCoCk, a GenTle inTroduCTion To STaTa (4th ed. 2014) (providing an introduction to Stata for researchers working in psychology, social sciences, and other fields that require quantitative analysis). 61 For more on the methods employed in the DLA Project, see Deo, Looking Forward, supra note 13. 62 IRB certification is on file with the author. Columbia Journal of Gender and law22 31.1 iii. Challenging Confrontations & Biased Comments DLA findings regarding racial/gender disparities related to teaching center on two particular themes: 1) challenges to classroom authority, and 2) biased teaching evaluations. The classroom is challenging for almost all new faculty. Even white men in the DLA study admit that the material and workload can be challenging, though few have disruptive students or negative evaluations wholly unrelated to their actual teaching. Women of color find it especially difficult to navigate a new teaching experience, a new job, often a new city, and the manifestations of race and gender discrimination from students in the classroom. This Part situates findings from the DLA study in the context of the law school environment, focusing primarily on the experiences of women of color law faculty as compared to their white male colleagues. All names listed are pseudonyms to protect the anonymity of actual DLA law faculty participants; all quotes used are the actual words of the law professors who participated in the DLA study. As Mariana, a Latina, notes of her first semester experience in the classroom: “It was rough.” Often that is because what/ whom the students expect—traditionally, an older, white, male law professor—does not match up to the young woman of color in the front of the room. As a Black female named Patricia says, “Their image is the white male professor who scares them, and that’s just not a model that I can follow. Not being a white male, and not really being privileged enough to be able to scare them.” Grace, a multiracial woman, agrees, noting, “I’m certain that they feel differently when I’m teaching than when the older white male professors [at my school do the same].” A. Classroom Confrontations “I had one student tell me once, ‘I never had a Black woman tell me anything who wasn’t dressed in white.’” –Trisha 1. Disappointing Students’ Expectations DLA data indicate that students often manifest the disconnect between the reality of the female faculty of color at the front of the classroom and the expectation of a white male professor standing there through blatant or subtle forms of disrespect in the classroom. Even when the disrespect is subtle, it nevertheless requires significant energy for the female law faculty of color to respond in a way that maintains control of the class without alienating the other students. For instance, a tenured Asian American law professor named Annalisa still feels that “in the classroom as a small Asian American woman I feel that I Columbia Journal of Gender and law 2331.1 have to establish authority in the classroom. I feel that my tall white male colleagues . . . have an easier job of establishing a sense of authority. And so it’s a challenge basically, but it’s not insurmountable.” Annalisa’s stature works against her with students who respond to her looks by correlating her small size to a lack of authority. Her positive outlook is shared by others, though many sense more disappointment or even outrage from the students in their class who were not expecting a woman of color to teach them. A Black female Associate Dean named Susan reflects back on one of her earliest experiences teaching first- year students and recalls their disappointment and sense of unfairness in being placed in her class, noting, “They thought they had the dud professor: ‘Oh my god, we got the Black lady teaching us and [the other Section] got the white guy?!’” Perhaps because of this mismatch between what the students expect and who is actually present, some students lash out in the classroom, with the female faculty member as their primary target. Erin’s experience during her first year on the tenure-track, as a Native American law professor at a predominantly white school, is not uncommon. She says, “I was heavily criticized my first year of teaching.” As part of an end-of-term skit performed on campus in front of numerous faculty and students, her students mocked her publicly by making “a lot of jokes about me having a chip on my shoulder or me having a stick up my ass, just being perceived as being very cold.” Clearly, the students felt comfortable acting out their notions about her, regardless of her status as their professor. Aisha, an Asian American professor, states, “I certainly have felt the presumption of incompetence” from the students, attributing it to students who have never before been faced with a woman of color authority figure. Gabrielle, a Black law professor, recalls direct confrontations from students during class, especially early on in her career, noting that “for my first couple of years they were openly disrespectful in the classroom.” A multiracial woman named Emma attributes the pushback to their disappointed expectations, noting: They don’t necessarily appreciate somebody in my position as a new professor, as a woman, and as a person of color having power over them. So I got a lot of push back particularly from male students about, “Do you know your stuff?” “Are you a credible source of authority?” “Why should I listen to this stuff from you?” When asked directly about whether there was a racial or gender pattern to the students who were challenging classroom authority, most faculty responded in a fashion similar to how Hannah, a multiracial tenured faculty member, responds: “It tended to be white men, absolutely.” A Black law professor named Patrice has a similar experience, noting that when she thinks of the students who initiate confrontations or challenges in the classroom, that may Columbia Journal of Gender and law24 31.1 fluster the professor and disrupt learning for fellow students, “they’re primarily white men.” 2. Examples of Confrontations In terms of the specifics of what these mostly white male students do, the examples run the gamut—from subtle challenges to overt physical intimidation. A multiracial faculty member named Emma says, “I would push people using the Socratic method to answer the question, [but] I’ve had students who refuse to answer the question and then put me in a spot where I have to enforce my authority that they need to answer.” Clearly, this is a direct challenge to her authority, with students simply refusing to follow her rules in class. Destiny, a Black female professor, notes that it tends to be how students respond to her even more than what they say specifically that signals their disappointment in having her as a professor and their resulting display of disrespect. Destiny says her students exhibit “a kind of tone and attitude that they just don’t have with some of my male colleagues.” She attributes it to “[a] kind of a disbelief [that I’m their professor], just saying things that they would not normally say. My students still get kind of [confrontational, with an attitude like] how dare you try to teach them, and can be challenging.” Other white male students subtly or directly point out their racial and gender privilege to the female faculty members whom one would expect to be the person with the most power in the classroom. One of the most poignant examples comes from Trisha, a senior Black scholar who entered the legal academy as an Assistant Professor over twenty years ago. Trisha makes clear that she has endured “comments in class, [and] outside of class,” many of which were “inappropriate.” One particular encounter stands out to her, which she remembers vividly though it took place a number of years ago: “I had one student tell me once, ‘I never had a Black woman tell me anything who wasn’t dressed in white.’” She speaks with gravity about her response at the time, “It took me a minute. I had to think about that, ‘What do you mean dressed in white?’ Okay so I got thinking like the roles he was assigning me were maid, nurse, I don’t know . . . .” Not only did Trisha’s student see her in that light, but he expressed the sentiment to her directly, likely attempting to put her in her place or the place he felt she belonged. While those sorts of allusions are insulting and tend to elicit emotional responses from the female faculty of color targeted, some faculty also endure horrific physical intimidation at the hands of white male students in class. When a multiracial woman named Grace was a new law professor teaching rape as part of the first-year Criminal Law curriculum, she encounter a particularly appalling example of white male privilege displayed in her class: Columbia Journal of Gender and law 2531.1 [A] group of white, kind of “frat boy” types . . . were sitting [together] in the classroom. One kid brought in a noose and in response to a question, so he must have planned this, he pulls a noose out from under his desk, holds it up, and says because he’s “well hung.” I am certain that would have not have happened to another [white, male] professor. It was so shocking. Natalie, a multiracial woman who entered legal academia over a decade ago recalls how gender and race contributed in a “huge” way to her early experience in law teaching, noting that she “was tortured by those standards. Oh the males, everything you can imagine how women are treated by men students, disrespected and everything like that. In the classroom, out of the classroom you name it.” Thinking specifically about her in-class experiences, she recalls one experience when a male student “came in forty-five minutes late to a fifty-five minute class and I told him to get out and he had me up against the wall with his finger [in my face] like how dare I kick him out of the class.”63 Lola, a Latina law professor, endured another common first-year incident for female faculty of color: a student who wrote a letter complaining about her teaching and submitted it to the administration. Lola recalls: I had someone go out of her way to write a very disgusting letter. [Though it was unsigned,] I know who wrote it. This is someone who came to my office four days after I started teaching and told me that I wasn’t from around here. And I needed to basically learn how things are done in [this state]. Thus, student confrontations against law faculty members in the classroom are serious and ongoing on many law school campuses. Though a separate article utilizing DLA data delves more deeply into faculty-student interactions,64 these data are presented here primarily to set the stage for the next Section. Biases in student evaluations, presented below using DLA data, must be analyzed in the context of the challenging relationships many female faculty of color have with some—primarily white and male—students. B. Biased Evaluations “Why doesn’t she wear her wedding ring? Is she trying to tease us?” –anonymous student evaluation quote, recounted by Natalie 63 For more on challenging interactions with students overall, see Deo, Ugly Truth, supra note 11. 64 Deo, Ugly Truth, supra note 11. Columbia Journal of Gender and law26 31.1 While both public and private conflicts can be challenging, precisely because of their confrontational nature, course evaluations give students a risk-free opportunity to convey their biases anonymously. In a sense, the challenging personal interactions in class set the tone for the biased teaching evaluations that follow at the end of the term. Though evaluations are supposed to be an opportunity for students to offer their observations, focusing on the pros and cons of the professor’s ability to convey the substantive material taught in each course, many women of color remember evaluation comments focused more on style or personal appearance, and often laced with both subtle and direct racism and sexism.65 Helen, an Asian American who entered legal academia a decade ago, uses a common adjective to describe both her early teaching evaluations and her emotional response to them: “terrible.” 1. Channeling Sofia Vergara Brianna, an African American scholar, says that teaching evaluations give students an opportunity to be “hypercritical”—and students do not censor themselves to offer comments only about how the professor approaches or conveys the legal doctrine. Trisha, a Black female, notes that her evaluations include many “inappropriate personal comments about my appearance,” in addition to the comment a student made to her directly (discussed earlier in this Article) about how, in his experience, Black women addressing him wore white and had inferior status. June, also a Black woman, remembers that early in her career there were multiple students who commented on her appearance in teaching evaluations and made clear that her personal style was their in-class focus as well, since one student evaluation comment “said, ‘I came to class [because] I wanted to see what you were going to wear.’” These evaluations did not contain content or comments on June’s pedagogical approach to the material she taught and it is unclear whether her students focused on the substance that semester; yet, many used their evaluations as an opportunity to provide feedback and critique—on June’s physical appearance alone. Annie, an Asian American senior faculty member, remembers that early in her law-teaching career “there were a lot of comments about my appearance. I think women tend to get that more than guys.” Carla, a Latina, is concerned about the students’ emphasis on her appearance, but does her best to approach this fraught reality with humor: As a woman there is a constant kind of student concern with my looks. I know my hotness factor is low [laughing]. Why? Why? I’m not supposed 65 For a more detailed discussion challenging the usefulness (or lack thereof) of teaching evaluations generally, see Lazos, supra note 29, at 167. Columbia Journal of Gender and law 2731.1 to be Christie Brinkley in the front of the room and yet, am I? Answer: No, I’m not! [B]ut that’s another element on which I get graded [on student evaluations]: am I “attractive.” In fact, social science research makes explicit the connection between positive teaching evaluations and (supposedly) “objective” measures of beauty.66 Researchers have discovered that professors who are “judged [to be] better looking” receive better evaluations from students, with an even more significant penalty for faculty of color who are deemed to be unattractive.67 As a result, women of color must devote time and attention to their personal appearance in the classroom, in a way that few other faculty members do, or risk even lower evaluations and scathing comments from students focused on style over substance. Thus, while Carla laughingly acknowledges that she does not look like an attractive female celebrity, much to the disappointment of her students, the research suggests that her evaluations might go up if she did channel Sofia Vergara. 2. Microaggressions and Cruel Comments Existing scholarship has already concluded that race and gender bias colors teaching evaluations.68 The DLA data reveal the extent to which comments on evaluations are blatantly discriminatory with regard to both race/ethnicity and sex/gender.69 Many women of color note that their raw scores on evaluations may be less positive than they expect and feel they deserve, but the comments reflect conflicting reasons for these low scores 66 Of course, even “objective” measures of beauty are heavily influenced by the white male norm in the United States. “The mainstream beauty ideal is almost exclusively white, making it all the more unattainable for women of color,” leaving female faculty of color in a paradox of sorts since they are judged even more harshly than white female faculty on their appearance. See Beauty Whitewashed: How White Ideals Exclude Women of Color, beauTy redeFined bloG (Feb. 1, 2015), http://www.beautyredefined.net/beauty-whitewashed-how- white-ideals-exclude-women-of-color [http://perma.cc/2FV8-HQCD]. 67 Daniel S. Hamermesh & Amy M. Parker, Beauty in the Classroom: Instructors’ Pulchritude and Putative Pedagogical Productivity, 24 eCon. eduC. rev. 369 (2005). 68 See Linda S. Greene, Tokens, Role Models, and Pedagogical Politics: Lamentations of An African American Female Law Professor, 6 berkeley women’S l.J. 71 (1990). 69 Most scholars working in this area relate the term “sex” to the biological features differentiating men and women whereas “gender” refers to the exhibition or presentation of oneself as either male or female. See, e.g., Leslie Bender, Sex Discrimination or Gender Inequality?, 57 Fordham l. rev. 941, 946 (1989) (explaining how sex generally refers to “certain physical attributes, in particular our primary and secondary sexual characteristics and life-giving capacities,” whereas gender is the “cultural/social construction and attribution of qualities to different biological sexes”). Columbia Journal of Gender and law28 31.1 or offer further negativity without constructive critique. For instance, Carla continues that in one particular semester, her evaluations included allegations that “I let students ask questions, [as well as that] I don’t let students ask questions—very illogical. The numbers were fine but the comments were vicious.” The inconsistency of the comments also calls into question the veracity of what is being written. Also, if the criticism were constructive, individual faculty members might be able to address legitimate student concerns; but when it is simply rude, cruel, or nonspecific, there is little benefit to anyone. Lola, a Latina, recalls that her first year she “got a lot of really inappropriate comments.” They were “very personal comments just unrelated to my teaching. That I was a disgrace. I remember that was one of them. That I was a disgrace to the school, with no basis [given].” Again, if the students had suggested that Lola were not well versed in a particular area of the law, or that they had a hard time understanding the doctrine due to her pedagogical technique, she could consider varying her approach to the material; but simply stating that she was a disgrace attacked her personally without providing input on how to improve in the future. Even female faculty of color who have excellent teaching evaluations in terms of the numbers receive racist/sexist comments from students. For instance, a multiracial faculty member named Grace notes that her evaluations “have been fabulous” at every institution where she has worked; yet, on occasion she does “get the random comments from students, like, ‘Professor Grace doesn’t like white people.’” June, a senior Black law professor, recalls that she had especially vicious comments on student evaluations “[e]arly in my career.” One comment in particular still stands out, twenty-five years later: “I remember this statement, ‘I know we have to have affirmative action, but do we have to have this woman?’ That was pre-tenure.” Again, this comment does not explain student challenges with regard to mastering the material or provide suggestions for how June may improve her approach; instead, it reveals stereotypes that the student holds about Black women with an attempt at stigmatizing the professor by labeling her as someone who was hired through affirmative action, in spite of her assumed incompetence. Similarly, an Asian American named Aisha gives the following examples from her evaluations, “‘Don’t know why she’s teaching.’ ‘She’s terrible.’ ‘You should have hired somebody else.’ ‘There was not enough law, there was not enough [state] law, too much social science,’” even when she exactly tracked the casebook. Natalie, a multiracial female, recognizes how anonymous comments can have subtle racial and sexual overtones, mentioning that her negative evaluations “run[] the gamut from ‘Why doesn’t she wear her wedding ring? Is she trying to tease us?’ to ‘She thinks she’s so smart.’” Columbia Journal of Gender and law 2931.1 Lola, a Latina law professor, reflects that her evaluations have “more micro- aggressions70 than blatant racist comments,” but she has plenty of student detractors who write nasty comments: “She’s not very smart.” “She’s not prepared.” “She’s hostile.” The “hostile” comment seems to come up at least once a semester on the evaluations. And it’s consistent with the comments that the Black professors [here] are getting. Lola is correct that she is not facing an individual problem—where students identify particular concerns about her teaching that she can address through careful response—but rather is on the receiving end of a larger structural problem where course evaluations are used as an avenue to denigrate female faculty of color. She herself was denied promotion recently in spite of the Dean telling her, “‘You’ve met all of the criteria,’” because the Dean “claimed that my teaching evaluations were a bit polarized.” This result directly exemplifies how even a few negative comments on teaching evaluations can have dire and long-lasting professional consequences for female faculty of color. In fact, Lola notes that the majority of her evaluations were positive with just a few outliers containing harshly negative comments. Yet, she believes that her senior colleagues had no other basis for denying her the promotion; she specifies: [T]hey couldn’t get me on scholarship because I published more than they require. They couldn’t get me on collegiality because I go out of my way to support my colleagues and whatever they’re doing for the school or personally. Thus, the only remaining mechanism for the administration to deny her promotion “were some racist, mean comments in my evaluations.” Though the experience of being denied promotion was “pretty brutal for me,” she is prepared for an upcoming tenure fight, making clear that “if I don’t get tenure next year, I’m suing the school.”71 70 Microaggressions are the “subtle verbal and non-verbal insults directed toward non-Whites, often done automatically or unconsciously.” They tend to be “layered insults based on one’s race, gender, class, sexuality, language, immigration status, phenotype, accent, or surname.” Daniel Solórzano et al., Keeping Race in Place: Racial Microaggressions and Campus Racial Climate at the University of California, Berkeley, 23 ChiCano- laTino l. rev. 15, 17 (2002). 71 In fact, tenure lawsuits came up frequently in the DLA data and are discussed in greater detail in subsequent articles discussing DLA findings. See, e.g., Deo, Mentors, Sponsors, and Allies in Legal Academia (unpublished manuscript) (on file with author). Columbia Journal of Gender and law30 31.1 Stacey, a Black female, also recognizes that she has “[reached] a point where I’m really kind of bitter and I’m just sick of it.” Partly because she is “the only person, or one of very few” at her current institution who encourages diversity discussions72 in the classroom, her doing so “stands out” and students accuse her of “hav[ing] an agenda, even if they like me. Oh, they like me, but they’re still unfair to me. You know, I can read it in their comments, and I see it in their numbers.” But because “my white colleagues don’t think about” how evaluations may be biased against women of color, she worries that they see her as less accomplished when her “numbers” (i.e., average scores from evaluations) are lower than she merits. This fear of being perceived as incompetent, and of responding to related microaggressions by overcompensating, is also common.73 In fact, one standard response to racial microaggressions, according to existing literature in this area, is “to ‘prove wrong’ the ideas, statistics, statements and attitudes that say People of Color are less intelligent or less capable than others.”74 While it might be an effective “strategy as resistance to a negative campus racial climate” for some female faculty of color, it comes at an exhausting cost of constant work coupled with the fear that no amount of success will be sufficient to overcome the negative stereotypes and structural biases working against them.75 Unsurprisingly, students who seek out and appreciate female faculty and especially female faculty of color in private meetings give much more positive feedback in course evaluations. As most evaluations do not track race/ethnicity, gender, or grade performance statistics of the students who complete them, there is little reliable information about which students leave which comments. Including some context, whether the racial/ethnic background of the student, her expected course grade, or even her gender, could be helpful in determining how useful the evaluations themselves actually are. Very occasionally, students will reveal some of this information of their own accord in their comments; when they do so, their appreciation in private meetings does seem to track their enjoyment of the course and the professor.76 For instance, an Asian American faculty member named Vivian 72 Diversity discussions are conversations about race, gender, sexual orientation, class, and other sensitive subjects that students appreciate discussing, but faculty often shy away from for fear of saying the wrong thing and appearing politically incorrect. Meera E. Deo et al., Paint by Number? How the Race and Gender of Law School Faculty Affect the First-Year Curriculum, 29 ChiCano/a-laTino/a l. rev. 1, 2–3 (2010). 73 See, e.g., Daniel Solórzano et al., Keeping Race in Place, supra note 70, at 42. 74 Solórzano et al., Keeping Race in Place, supra note 70, at 42. 75 Solórzano et al., Keeping Race in Place, supra note 70, at 43. See also infra Part IV.A on individual responses to these challenges. 76 For more on how students from all racial backgrounds may respond more positively to female faculty and faculty of color in the classroom especially, see Deo et al., supra note 72. Columbia Journal of Gender and law 3131.1 recalls that she was “shocked” after her first teaching experience as she “was reading these evaluations saying, ‘Oh, she’s a terrible teacher,’ and ‘I didn’t learn anything.’” She then read another evaluation in the same batch “from a student saying, ‘I am a woman of color. This is the first class that I’ve taken at this law school that has made me feel like a human being.’” Though the negative evaluations did not specify the race or gender of the writer, many were likely from white students who comprised the majority of students in her class—likely many of the same students who challenge their faculty of color in class. C. Contrasts “[M]y teaching evaluations have always, consistently been very strong.” –Joe All of the above are samples of the current biases in teaching evaluations, the very materials used to determine, in part, whether faculty members have satisfied the requirement of excellence in teaching in order to receive tenure. While the illustrations provided thus far have been of challenges facing female faculty of color, who are the central focus of the DLA study and of this Article, some white female and men of color faculty confront challenging classroom dynamics and bias in student evaluations as well. White male faculty tend to have a much different experience. 1. Other “Outsider” Experiences For instance, many of the white females in the DLA sample confront similar challenges to women of color, with white male students acting out in class and challenging their authority, and receiving teaching evaluations that focus on appearance and other non- teaching related issues. For instance, a white female law professor named Chloe recalls that her first year teaching she dealt with numerous student issues involving “who’s in control of the class.” These challenges to Chloe’s classroom authority parallel what many of the Black, Latina, Asian American, Middle Eastern, and multiracial women faculty members face. Similarly, a white female law professor named Marybeth recalls that her first year teaching, “there were some guys in the class who thought I had it in for them,” which resulted in a difficult year for her where “every bad thing that could happen in law school class happened to me in that classroom,” in spite of the fact that she considers her evaluations overall as “pretty good. They are good.” Yet, while some challenges apply universally to women, regardless of their racial/ethnic background, the highest hurdles are placed before female faculty of color alone. White male law faculty, who are the largest intersectional race/gender group in legal academia, have a very different experience. Columbia Journal of Gender and law32 31.1 A Black professor named Susan contrasts her extraordinary efforts in reviewing all relevant material to avoid having to tell a student that she did not have the answer to a question, to those of her white male faculty colleague Tim77 who started the same year as she did. She recalls Tim’s response when she mentioned her strategy to him: Tim was like, “You know, gosh I would definitely have said, ‘I don’t know, I’m going to have to get back to you on that,’” which is the privilege of being a white man; you can say, “I don’t know,” and its not gonna damn you to being, you know, the ignorant law professor. But I don’t have that privilege. Instead, if Susan were to admit to not knowing something, she would be confirming existing stereotypes that many students hold about her even before classes begin: that she is incompetent, that she is an “affirmative action hire,” that she is not fit to teach them the law. A white woman named Ellen notes how the presumption of incompetence works against women of color, even when compared to their white female colleagues. Ellen remembers that when she taught Property at a previous institution, “a woman of color was also teaching Property, [using the] same book; our syllabi were identical and our coverage and approach was the same.” The main difference between them from what the students could see is that Ellen is a white woman, while her colleague was “an African American woman with natural hair.” Yet, Ellen’s colleague was treated very differently by the students: She got comments all about her “crazy civil rights agenda” on her student evals. I didn’t, not one. Did we both cover the Civil Rights Act? Yes. Did we both touch on issues of race and issues of privilege in a whole lot of different ways? Yes. Ellen still believes it is “easier for me” as a white woman to bring up issues of race and ethnicity with her students than it is for a woman of color, since she is “familiar with all the literature essentially saying that when you do this it’s not part of your own agenda because these are not ‘your own people’ or ‘your own concerns.’” Ellen’s experience that semester, especially in comparison to her colleague’s challenges, “really reinforced my commitment to [incorporating diversity discussions in the classroom],” in part because she knows how heavily her female faculty of color colleagues are penalized for doing the same. 77 Pseudonym used to protect confidentiality. Columbia Journal of Gender and law 3331.1 Ryan, a Black male law professor, has enjoyed a positive experience teaching, though comments in his consistently stellar evaluations sometimes contain elements that allude to existing racial and gender bias. Ryan notes that in the five years that he has been at his current institution “I won [teacher of the year] twice.” Students do not directly confront him or challenge his authority in the classroom or in private meetings. In addition, Ryan regularly receives excellent teaching evaluations, which he notes are “anywhere between 98–100% in just about every category.” Nevertheless, there are some anonymous comments on the evaluations that accuse him of “picking on the police,” or calling him both “a racist and a sexist.” One student mentioned that she was a woman and said Ryan “made her afraid in the classroom,” perhaps subtly inferring some combination of racism and sexism herself.78 Thus, it is not only women of color who face challenges in the classroom, but others who are traditional outsiders in legal academia. This outsider status may be based on race, gender, sexual orientation, socioeconomic status, and intersectional combinations of various identity characteristics. For instance, a white lesbian named Sydney has had more than her share of disrespectful students, as well as negative and irrelevant comments on her teaching evaluations. She notes, “[M]y evaluations included mean things about why it’s ‘offensive’ the way that I dress, [such as,] ‘She makes herself gender neutral and that’s offensive.’” Of course, it is unclear how Sydney’s choice of dress and personal style are relevant to teaching effectiveness, let alone how presenting oneself as gender-neutral could be offensive to students. Yet, these are the types of comments that Sydney endures. She knows that the comments are “due to my gender and [sexual] orientation and the way that [students] responded to those things.” 2. White Male Experiences White men, especially those perceived as conforming to the expectations of their students with regard to their other identity characteristics, rarely face such hurdles. When recalling his success in the classroom, a white male professor named Joe notes that “my teaching evaluations have always, consistently been very strong.” This is true of virtually every white male professor in the DLA sample, again in stark contrast to the female faculty of color in the sample. For instance, a white male Assistant Professor named Justin characterizes his teaching evaluations as “on the whole very positive,” in spite of the fact 78 A few men of color face challenges in the classroom that parallel those experienced by female faculty. For instance, a Black man named Michael recalls that he “had a student who would routinely show up five or ten minutes late and he would have to walk behind me to get to his chair. . . . Or students who answer their cell phones in class.” Columbia Journal of Gender and law34 31.1 that he thinks he “come[s] to class a little scatterbrained.” When pressed about his first semester teaching law, he notes that he taught Criminal Law to a group of students who “cut me a lot of slack, so it was nice and a lot of fun.” The students cutting him slack and his clear enjoyment of the teaching experience even in the first semester are at odds with the experience of the vast majority of the women of color in the DLA sample. Another white male professor named Ian also has generally positive teaching evaluations. Thinking generally about his student evaluations, he states, “The general themes are I’m one of the most challenging professors and they appreciate that in terms of the subject and expectations I set.” Interestingly, his students note that he is challenging as a professor, but do not complain or push back against the high expectations he sets; instead, they appreciate him for it. A white male DLA participant name John is also representative of others from his race and gender. John’s evaluations from students are “[r]eally strong. On a five point scale for numerical [values] I’m consistently above a four and [above] the college average, usually in the 4.5s. And the written comments are almost always very positive.” Again, the experience of these white men is in stark contrast to those of their women of color colleagues, most of whom endure challenging interactions with students in the classroom and negative comments on teaching evaluations. iV. Responses to Student Bias Studies have documented how teaching evaluations are biased against women faculty as a whole and women faculty of color in particular.79 Interestingly, it is unclear whether course evaluations have any actual value, as they rely on students who are by definition not well versed in the subject matter, evaluating faculty who know a great deal on the substantive topic of the course.80 In fact, the comments on many student evaluations bypass substance altogether to focus wholly or primarily on the style, appearance, racial/ethnic background, gender, or other personal characteristics of the instructor. Recognizing these severe limitations, perhaps it is time to do away with course evaluations altogether, or at least to revise either the format or the weight they are given for promotion and tenure.81 Marisol, a Latina law professor, proposes that law schools at least approach evaluations with some measure of skepticism; she suggests that when administrators rely on evaluations for tenure or promotion decisions, they should first question whether bias could be playing 79 See, e.g., Lazos, supra note 29, at 167. 80 Lazos, supra note 29, at 167. 81 For more on the problematic nature of teaching evaluations, and especially the race and gender ramifications, as well as concrete suggestions for improvement, see Deo, Ugly Truth, supra note 11. Columbia Journal of Gender and law 3531.1 a role. She has seen “not only from my perspective, but [also from] looking out there and watching other people,” that “people of color don’t get tenure as much because they’re not supported” in the classroom or with their research. Just as Lola was denied promotion because of a few negative comments on evaluations, many others have faced similar circumstances. Marisol suggests that once faculty of color are hired, administrators should “try to understand what other things are going on and [ask,] ‘How can we protect this person so that they can grow as a teacher and a scholar and get tenure in the future?’” Instead, with student evaluations currently given full credence and weight by many faculty administrations without any questioning or consideration of possible bias, Marisol believes that “that’s where they ‘get’ a lot of people [of color],” who start off excited to be hired into legal academia but are ultimately denied the security of tenure because of negative student reviews. The DLA data reveal this directly, with the experience of Lola and others whose negative student reviews were cited as reasons their promotions were denied.82 With these challenges in mind, this Part presents both individual strategies that many DLA participants have utilized themselves and broader structural solutions that administrators and institutions must adopt if they seek to fight bias in teaching evaluations. A. individual Strategies “I responded by staying ten steps ahead of them. . . . Because I knew that when that day came that I said, ‘I don’t know,’ I would lose all credibility in the classroom.” –Susan Many of the women in the DLA sample employ specific tactics to avoid direct confrontations to their authority and hopefully mitigate the negative teaching evaluations that invariably appear at the end of the term. While none of the DLA participants admitted this directly, perhaps if they had distributed chocolate, as one researcher has suggested as an effective mechanism for receiving positive student ratings, their evaluations would have improved overall.83 Instead, many DLA participants offer concrete suggestions and practical advice for classroom management and improving teaching evaluations. Others avoid reading them altogether. 82 See supra Part III.B.2. 83 See Lazos, supra note 29, at 170 (“If all else fails, give your students chocolate before handing out the evaluations.”) (citing R.J. Youmans & B.D. Jee, Fudging the Numbers: Distributing Chocolate Influences Student Evaluations of an Undergraduate Course, 34 TeaChinG PSyChol. 245 (2007)). Columbia Journal of Gender and law36 31.1 1. Avoidance A number of faculty members follow Natalie’s lead with regard to course evaluations. Natalie, a multiracial female law professor, says, “I stopped reading student evaluations when I got tenure. I don’t read them. I refuse.” Danielle, a Black woman, is unapologetic about it, saying, “I don’t even read my evaluations anymore. I don’t read my evaluations because I know I’m being evaluated based upon things that do not have to do with my teaching ability.” She elaborates, saying, “I got an evaluation that said . . . , ‘She’s Black, enough said.’ So I’m being evaluated based upon things that have nothing to do with my teaching, like my skin color.” When it’s clear that the evaluations tend not to comment on the actual curricular or pedagogical approach, but focus on race, ethnicity, personal style, sexual orientation, gender, or other topics irrelevant to learning, some faculty members decide to simply avoid them. Sydney, a white lesbian professor, notes that avoiding the evaluations does not mean these faculty members are not interested in improving their teaching; they simply recognize that it may be more important to protect their emotional health, especially if the evaluations are unlikely to contain valuable feedback on teaching itself. Sydney says, “[T]he experience of reading [evaluations is] so bad and that’s why I haven’t read my evaluations since [early on]. Which isn’t to say I don’t want to improve my teaching, I care a lot about that. But I don’t think evaluations are the way.” 2. Commanding Authority Other faculty members do continue to read their evaluations and actively seek out ways to improve them, avoid student confrontations in class, and command the authority and respect they deserve. For instance, a Black woman named Susan recalls how she was challenged on a daily basis by students who would badger her with questions about the material in an attempt to trip her up or show that she was not adequately prepared. Instead of getting frustrated, disengaged, or giving up, Susan says: I responded by staying ten steps ahead of them. So I not only read the law review articles cited in the notes after the cases in the casebook, I read the law review articles cited in the law review articles so that any question that they asked me I could respond. Because I knew that when that day came that I said, “I don’t know,” I would lose all credibility in the classroom. In other words, Susan knew there was a presumption of incompetence working against her even before she entered the classroom. Though it took significant time and effort to combat Columbia Journal of Gender and law 3731.1 it, she chose to invest in reading everything she could to avoid confirming the students’ stereotype that she was not prepared to teach them. Eliana, a Latina Assistant Professor, notes that she never had problems with students confronting her or challenging her authority in the classroom, “but that’s because I’ve spent so much time anticipating” and strategizing about how to avoid it. Here’s how Eliana describes her plan: I immediately come in with a lot of authority, with a lot of telling [them], “These are my credentials.” “This is why we are doing this.” But that’s extra work, right? You have to think about that versus a white male colleague [who only has to say], “Oh, we are going to talk about this topic today.” It takes a lot of energy [for me]. Eliana is a mother and draws on the skills she has learned as a mother in the classroom, noting that while she will “show them immediately that I’m not afraid to engage with you,” she also uses “a lot of skills of like being a mother: very sweet, very nice, very nurturing, but switching at a moment’s notice and letting them know there is no messing around in my class and these are my expectations.” In fact, she thinks her approach is a “feminine” one, focused on relationships and making clear her expectations and her encouragement so that her students “don’t want to let me down.” Eliana has employed what has been termed “gender judo” by finding a way to use characteristics that should work against her—young, single, mother—to her advantage in the classroom.84 At the other end of the career spectrum is Michelle, a Black woman who entered legal academia over thirty years ago and now holds an endowed Chair at her law school as well as a university-wide administrative position. Remembering her early days in the legal academy, Michelle says: I just decided I wasn’t going to let the students make me lose my cool in the classroom. It was my classroom. Like [one student in class] said to me, “I just think you are wrong.” Yeah. I take a moment; I go to the 84 For a discussion of Eliana’s approach as a utilization of “gender judo,” see Deo, Ugly Truth, supra note 11, at 1007 (“gender judo” has been defined as “the purposeful decision to ‘take feminine stereotypes that can hold women back—the selfless mother and the dutiful daughter, for example—and use those stereotypes to propel themselves forward’”) (quoting Joan C. Williams, Women, Work and the Art of Gender Judo, waSh. PoST (Jan. 24, 2014), http://www.washingtonpost.com/opinions/women-work-and-the-art-of-gender- judo/2014/01/24/29e209b2-82b2-11e3-8099-9181471f7aaf_story.html [http://perma.cc/S6JX-XNEP]). Columbia Journal of Gender and law38 31.1 board, erase the board to give myself a few minutes to calm down, and then I would sort of turn back around and smile and say, “That’s really interesting Mr. So-and-So. Tell me more, tell me more.” And I would do it in like a very personable way. You know I’d smile like, “Come on, tell me more. Come at me, come at me, tell me why I’m so wrong.” Part of Michelle’s purpose for doing this was to put the other students in the classroom at ease, to bond with them, in a sense, or prevent their automatic bonding with the student who had confronted her. She remembers that at the initial challenge, when a student would tell her she was wrong: the other students would become so tense like, “Oh my goodness, what’s she going to do?” And I learned if you ever yell or go after a student in the class, the other students will bond with that person because even though you don’t experience it that way, they experience it as someone with power going against someone without power. Instead, by encouraging the confrontational student to explain why Michelle was wrong, and with friendly body language (“I put my hand on my hip and I kind of smile at them”), she “would break the tension, other students would laugh, that [confrontational] student would become a little bit isolated, then I’d get pretty much like a hem and haw [from the student. At that point] everyone knows that they are not going to make me crack.” Again, Michelle had to employ a detailed strategy of forcibly calming herself down, then responding to the confrontational student in a seemingly friendly and non-threatening way, to not only break the tension, but also signal to all of her students that they would not be able to trip her up. Imani, a Black law female professor, also utilizes a particular strategy of starting off tough and easing off as the semester continues, so the students know that she will not tolerate confrontations or challenges to her authority. She says: I also am very tough, so I come in with a very high bar of no nonsense, if you’re not prepared that counts as an absence for me. So on the very first day I kind of come off as being somewhat intimidating, as students say. Now, I always dial that back, but I find as a female and a female of color it’s better to start out tough like that and dial it back. Because if you start out lax you can never dial it up. I don’t even want to open the door for that kind of disrespect to sneak in. Columbia Journal of Gender and law 3931.1 As a result, disrespect is less likely to infiltrate Imani’s classroom. Susan, who tries to stay ten steps ahead of her students, admits that in her first few years of teaching, she “got the worst teaching evaluations of anyone who had ever taught at the law school.” Because “of course it was” a requirement for tenure at her school that she demonstrate excellent teaching, she devised a strategy to combat the negative student evaluations: “I invited all of my colleagues to come and sit in on my classes— unannounced and as often as they would like. And I got excellent evaluations from all of my colleagues.” She made clear that they had no need to tell her in advance that they would be there, “because I didn’t want them to say, “Oh, well, you knew we were coming so you prepared extra well.” This created what she calls a “disconnect between the [negative] student evaluations and [the overwhelmingly positive faculty reviews].” Of course, this created the extra pressure of never knowing whether any faculty colleagues would be in her classes, but it was worth it for Susan. She says, “I’m confident that having [the peer evaluations] as the counter balancing narrative in my tenure file was the only way I got promoted.” Similarly, a Black female professor named Gabrielle will be forever grateful for “the support” of her school’s leadership early in her career, “when I was getting not-so- great reviews [from students] because people weren’t happy to have me as a professor.” To provide a better evaluation of her teaching effectiveness, the administration “sent faculty members in to review my class”; their reviews of her teaching, based on their many years of teaching experience and as more experienced colleagues, could therefore “counteract whatever students might say” on their anonymous evaluations. B. Structural Solutions “Do your evaluations suck because you suck as a teacher, or is there something else going on?” –Marisol Marisol’s quote above highlights a particular point stressed throughout the data presented in this Article: that negative teaching evaluations may have nothing whatsoever to do with teaching effectiveness, but instead be about the race, ethnicity, gender, or sexual orientation of the professor. In response, this Article proposes three possible alternatives to the current teaching evaluation fiasco. The first is to eliminate student evaluations altogether. Based on the literature showing that student evaluations are highly biased, not directly correlated to teaching effectiveness, and the clear bias in teaching evaluations that the DLA study reveals, there seems no clear purpose in keeping student evaluations. Student evaluations should be scrapped. Columbia Journal of Gender and law40 31.1 A second alternative is to keep student evaluations but modify them. One change would ask students to self-report their race and gender. Even if these characteristics were redacted from the comments presented to the individual faculty member assigned to the course, they might give those evaluating each candidate an idea of any racial or gender- based biases in the evaluation and signal to the full faculty that race and gender matter in student evaluations. The actual questions on the evaluations, which were not discussed in depth in this Article, could also be modified if necessary to more directly reflect teaching effectiveness, the professor’s ability to discuss sensitive topics of race, gender, class, and sexuality, or to be generally inclusive of all students. A signal from the administration that these goals are important could themselves improve the situation for female faculty of color, many of whom do these things quite well.85 A third proposal would replace (or supplement) student evaluations with peer evaluations. Most law schools do incorporate peer evaluations into the tenure file; thus, this proposal would not necessarily lead to any increased work on the part of faculty. In fact, it would result in less administrative work because student evaluations could be eliminated altogether, with no need to administer, analyze, or consider them for tenure or promotion purposes. Since peers are in a better position to evaluate their colleagues’ teaching effectiveness anyway, this seems a better fit than student evaluations with regard to determining the promotion viability of the professor being promoted. Schools could even solicit external reviews of all or particular courses, mirroring the ways in which scholarship generally undergoes review not only from faculty from within the applicant’s institution but also from colleagues at other institutions working in the same field. Thus, an applicant for tenure might have a tenured professor from her school sit in on one of her Civil Procedure classes and a tenured professor from a neighboring school sit in on another class.86 For law schools geographically isolated from all others, external teaching reviewers could either visit to observe class, or classes could be video- or audiotaped for other scholars to review at their leisure without leaving their institutions. Peer evaluations would likely offer much more useful critiques than student evaluations, 85 See, e.g., Deo et al., Paint by Number?, supra note 72, at 1. 86 While some might worry that institutional loyalty or competition would cloud peer evaluations, one hopes that individual law faculty members would rise above this when individual tenure or promotion decisions are at stake; in any case, candidates’ ability to have some input in the outside reviewers selected to observe their class could mitigate this if it does exist. Columbia Journal of Gender and law 4131.1 while also avoiding irrelevant topics like personal dress and marital status.87 Any additional biases could be mitigated by faculty candidates for tenure and promotion suggesting reviewers for class observation, just as most do for external review of scholarship. CONCluSiON Teaching evaluations are subject to bias, and female faculty of color experience bias in the classroom as well, especially through direct or subtle challenges to their authority. In fact, research indicates that students are more likely to reward instructors with positive evaluations when the teachers are highly charismatic, enthusiastic, or seductive—even if they spout gibberish and have little substantive knowledge of the material they teach.88 Since teaching evaluations are not only biased, but potentially useless, their usage for tenure and promotion decisions should be reconsidered. Of course, if teaching evaluations are useless measures of teaching effectiveness, that would hold true not only for female faculty of color, but for all law professors; however, the role evaluations play in tenure and promotion decisions becomes “particularly dangerous for minorities and women who must also contend with the unconscious biases of their students who have role expectations that are anchored in gender and race stereotypes.”89 Underrepresented law faculty are routinely challenged in the classroom and complained about in teaching evaluations. Ironically, these are the very faculty that administrators should be most willing to support, or they risk losing the few who are in legal academia. Even lower retention rates of female faculty of color would leave institutions and the legal academy as a whole even more homogeneous.90 How, then, can administrators support female faculty of color to avoid or ameliorate biases in student evaluations? This Article proposes three concrete solutions, any one of which would improve the current student evaluation fiasco: eliminate, modify, or replace/supplement. 87 Yet, there are documented biases among law faculty colleagues, particularly against female faculty of color. See Deo, Ugly Truth, supra note 11. Thus, even this solution is not fool-proof. 88 See Lazos, supra note 29, at 165 (discussing “lecturer charisma”) and 168 (discussing “educational seduction”). 89 Lazos, supra note 29, at 173. 90 See supra note 15, showing that women of color in legal academia account for only 7% of law professors in any capacity (i.e., tenured, tenure-track, visiting, adjunct, etc.). See also Katherine Barnes & Elizabeth Mertz, Is It Fair? Law Professors’ Perceptions of Tenure, 61 J. leGal eduC. 511, 512 (2012) (“[P]revious research demonstrated that these traditional outsiders were leaving the law school tenure track in greater proportions than were white men.”) (internal quotation marks omitted). Columbia Journal of Gender and law42 31.1 Option one would be to eliminate student evaluations altogether. Evaluations are not particularly useful measures of teaching effectiveness; instead, they seem both prone to bias and vehicles for discrimination. Biased comments can have real, negative effects on the tenure and promotion of already marginalized and underrepresented law faculty, particularly the women of color who are targeted for baseless remarks.91 Thus, it may make sense to discontinue the usage of evaluations, especially for tenure and promotion purposes. As an alternative, law schools could continue to use student evaluations but modify them to be both more useful and less prone to bias. The questions themselves can be modified, to make clear that students take an active role in their own learning.92 Questions can also be added to track the race/ethnicity, gender, and even the academic performance of students, both because students of color and female students may be more appreciative of particular faculty members who share their background and because white men may be more likely to be biased against these same individuals. Academic performance may reveal that struggling or unprepared students use evaluations as a means of lashing out, penalizing the professor for the student’s unprofessional attitude and behavior. A third option would be to replace or supplement student evaluations with peer evaluations. Many schools rely on peer observation in addition to student reviews to gauge teaching effectiveness for tenure and promotion purposes. However, outside of a formal review, it is uncommon for peers to evaluate colleagues’ teaching. Peer evaluations would clearly be more accurate measures of teaching effectiveness than student ratings, as faculty colleagues tend to know the material better and often appreciate diverse pedagogical and curricular approaches to learning.93 Peer evaluation could also include input from faculty 91 Other DLA articles have documented how women of color, who represent only 7% of all legal academics, endure challenging relationships with colleagues and students (see generally Deo, Ugly Truth, supra note 11) while also facing discrimination in hiring and advancement (Meera E. Deo, Trajectory of a Law Professor, 20 miCh. J. raCe & l. (forthcoming June 2015)). 92 More useful evaluations would focus on measuring objective outcomes for students, rather than their subjective assessment of the instructor. They would also include student responsibility for learning rather than focusing exclusively on the instructor’s teaching. For instance, a form might ask, “The instructor’s teaching techniques stimulated intellectual growth,” when it would be more useful to ask, “I developed my analytical skills in this course.” Similarly, the following quote puts active learning squarely on the professor’s shoulders: “The instructor created a classroom atmosphere that encouraged students to actively participate in their learning,” while active learning could be better measured with the following: “I participated in class by answering questions or engaging in discussions.” For illuminating exchanges on this topic and the actual examples given above, the author thanks Thomas Jefferson School of Law Dean Thomas Guernsey. 93 Others have also suggested peer evaluation, but it is generally proposed as a means of evaluating teaching Columbia Journal of Gender and law 4331.1 colleagues at local or even distant schools that have access to video or audio of class instruction. Just as outside reviews of scholarship provide input from experts in various fields from schools around the country, instruction could be peer reviewed and valuable insights gained with less opportunity for discrimination, especially if each candidate has some input in selecting the outside reviewers, as is the case for most outside reviews of scholarship. Any of the three alternatives to continuing to use student evaluations would be especially helpful because faculty goals for the curriculum are often at odds with student expectations, even with regard to faculty competency and effectiveness.94 Clearly, there is a disconnect between what the professor sees as important for learning in the classroom and what students think makes an effective teacher. Supplementing or replacing student evaluations with peer evaluations would therefore keep the emphasis on more widely accepted and established teaching goals. Of course, even if these proposals are adopted, not everything will immediately resolve for women of color faculty. Grace, a multiracial law teacher, states that as a woman of color, you’re “[a]lways having to prove yourself to groups of people. That you are smart enough, that you do belong there. I don’t care as much now because I’m post-tenure, but I do [care]. It is something that is always in the back of my mind.” So though these solutions cannot resolve everything for everyone, they would at least improve the situation for pre- tenure female faculty of color so that perhaps they can continue in legal academia, to fight a different tenure battle in the future. * * * effectiveness for promotion. See, e.g., Stark & Freishtat, supra note 40 (discussing the importance of including peer evaluations around “milestones”). 94 Wines & Lau, supra note 30, at 178.