STUDENT PERCEPTIONS OF EFFECTIVE TEACHING BEHAVIORS Developments in Business Simulation & Experiential Exercises, Volume 8, 1981 133 STUDENT PERCEPTIONS OF EFFECTIVE TEACHING BEHAVIORS George E. Stevens and Sheila Adams, Arizona State University Faith W. Stevens, Camden Community College INTRODUCTION The most common performance appraisal device used in academia today is the student evaluation of teacher effectiveness. Typically, the student, near the completion of a course, fills out a standardized evaluation form for the teachers of the class in which h or she is enrolled. Evidence of the widespread use at teacher evaluation has been documented by a number of researchers (e.g., Lein & Merz, 1978; Peterson, Kerin & Martin, 1978). For example, in a study designed to identify the methods used for evaluation of business faculty, Lein and Merz (1978) received responses from 374 business schools. Although respondents indicated that they used various combinations of methods in evaluating business faculty, over 70 percent of the schools used some form of teacher evaluation by students. Not only are more schools using this method of assessing teacher effectiveness, many also use the results to make administrative decisions (e.g., faculty retention, promotion, salary and tenure). As usage of student evaluations of teacher performance has increased, so has the amount of literature reporting the uses and abuse of these devices (see, for example, Miller, 1978; and Miller, Brokaw, & Shaaban,1977). It is evident that there are both proponents and opponents of the use of student evaluations of teachers as input into personnel decisions. Most faculty members agree that these evaluations have value if used for faculty development purposes but are leery of their usage for other purposes. One reason for this concern is the many reliability and validity issues related to teacher evaluations--issues which have been investigated by a number of researchers. Researchers have discovered, for example, that many who construct such ratings are not sufficiently qualified to do so (Costin, Greenough & Menger, 1971). Furthermore, when colleague and supervisor ratings of teacher effectiveness were also obtained, low correlations were found between colleague or supervisor ratings and student ratings, other researchers (e.g., Rodin & Rodin, 1972) conclude that students are not able to judge teaching effectiveness. Many variables have been identified which influence student perceptions of teacher effectiveness. In many cases, either the teacher cannot control the variables or the variables may be difficult to measure. Studies undertaken include those examining student attributes such as student achievement (Banziger & Smith, 1978; Costin, et al., 1971), achievement factors (Banziger & Smith, 1978); personality traits (Warren & O’Connell, 1978) ; and sex of student examined including leader behavior or style (Swanson, 1975; Kinicki & Schriesheim, 1978; Baba & Ace, 1978), type of course, i.e., required vs. elective (Miller, 1978; Miller, et al. 1977), course content, i.e., nonquantitative, primarily conceptual such as organizational behavior and marketing to more quantitative, less conceptual such as finance and operations management (Neely & Schaffer, 1979), teacher demands (Sullivan & Skanes, 1974), class size (Miller, 1978; Miller, et al. , 1977), sex of teacher (Elmore & La Pointe, 1975; Wilson & Doyle, 1976), and teacher personality (Elmore & LaPointe, l975; Witty, 1947). Although full discussion of these issues is beyond the scope of this paper, the interested reader is directed to see reviews such as those of Costin, Greenough, & Manges (1971) or Sullivan & Skanes (1974). A recent study by two of the authors (Stevens & Marquette, 1979) examined differences between faculty and student ratings of the importance of teachers’ course- related traits. If differences do exist in terms of faculty and student ratings of teaching effectiveness, then the potential value of student evaluations may be severely limited. Cummings and Schwabs’ (1973) findings suggest that supervisors and subordinates tend not to agree as to the dimensions of the subordinate’s job and the relative importance of the job dimensions. Furthermore, Maier, Hoffman, Hoover & Read (1961) found substantial disagreement between manager- subordinate pairs on the subordinate’s job duties and job requirements. One result of the Stevens and Marquette (1979) study was the finding that both students and faculty disagreed with the statement that they “used the same criteria to evaluate performance.” The researchers concluded that differences do exist between student and faculty perceptions of important teacher traits. That such differences do exist would seem to imply the need for caution in using the results of ratings in making personnel decisions. Beyond such a caveat, however, the question arises, do students themselves agree on the importance of course related teacher behaviors? Are there differences on the basis of the students’ sex when one examines a selected group of these teacher behaviors? What characteristics of teachers do students consider important to effective classroom instruction? In an attempt to answer these questions, the present study asked students to rate 17 traits commonly regarded as characteristics of effective teaching in courses where the teaching mode is predominantly lecture. METHOD Sample and Procedure A convenience sample of 257 students enrolled in a New Jersey community college was used. Responses were received from 145 female and 112 male students. Questionnaires were administered to the students by a faculty member during regular class time. The questionnaires were accompanied by a cover letter assuring respondents of complete confidentiality. Of the total number of students completing questionnaires 70 percent were business majors and 30 percent were non- business majors. Seventy-eight percent of the males and 63 percent of the females were business majors. (Table 1) Seventy-seven percent of the total sample were unmarried including 84 percent of the males and 72 percent of the females. (Table 2) Students in the sample were relatively young; 83 percent were 30 years old or less at the time of the survey, and 97 percent were 40 or less. A little more than half of the women were under 21 (55%) while nearly two-thirds (63%) of the men were in this age group. A somewhat larger percentage of the women (23%) than the men (9%) were over 30. (Table 3) More of the students were classified as freshmen (35%) than any other category and more of the women students were first year (44%) than the men (24%). (Table 4) Although of little relevance to the present study, the fact that such a large percentage of the females were freshmen coupled with the larger percentage of women over 30 than men of the same age may be related to the Developments in Business Simulation & Experiential Exercises, Volume 8, 1981 134 phenomenon of women returning to school and the work force. It is also possible that these facts are a reflection of the growing upward mobility in the female work force. Instrument The original questionnaire was composed of two parts. For purposes of this report, however, data from only the first part “Traits will be analyzed. This part of the instrument contained 17 statements describing teacher traits. Respondents were asked to indicate on a scale of 1 to 7, how important they believed each trait to be. (See Appendix A for copy of this part of instrument.) Demographic data were also requested indicating academic major, sex, age, grade point average, marital status, class and hours worked per week. RESULTS Results were computed for females and males as well as for the total group. Mean scores of both student groups were calculated and compared for each of the 17 traits. Comparison of these mean scores with t-tests was done and will be reported in the final version of the paper. Table 6 shows the 17 traits listed in the order of their importance ranking by the whole sample. The last two columns indicate how each trait was ranked by females and by males respectively. The rankings in Table 6 were arrived at by weighting each scale position by the number of students who selected that position. For example, if all females had rated the first trait-- Tests related to course materials--as extremely important (number 7 on the scale) then the value for females of that trait would be 1015, (7 times 145 female students). The actual value for female rating of that trait was 961. There were 112 female students who rated this trait as 7 in importance, 21 as 6, 8 as 5, 2 as 4, 1 as 3, 0 as 2, and 1 who failed to rate It at all. Multiplying the number of students times the rate and summing the results yields the 961 score. After such scores were calculated for each trait by females, males and total, the traits were ranked in descending order or scores. One question of interest to this study was which characteristics of teachers are considered important by students. The ranking of traits seems to yield a clue to the answer. Note the first three items in the table. Tests related to course materials, “clear expectations’ and “fair tests may all be said to have a clear, direct impact on a student’s grade in the course. Similarly, “requiring creative thinking, emphasizing factual knowledge and expecting students to be well prepared, (ranked 17, 14 and 13 respectively) may be considered to have a direct impact on learning. Having ‘high grading standards clearly impacts grades and it is ranked next to the bottom by the entire sample, by females and by males. Thus the evidence from this study would indicate that these students are placing more importance on grades than on learning. Some of the rankings unexpectedly contradict popular wisdom. For example, one would expect students (perhaps especially community college students) to consider applications of material more important than concepts. However, both males and females ranked emphasizing concepts” higher than “stressing applications’ although the difference in rankings was greater among the females (4 and 9 respectively) than among the males (6 and 8 respectively). Another such unexpected outcome was the relatively low rank (9) given to “exhibiting concern for students as individuals”. Despite the widespread expectation on the part of instructors that students expect a “dog and pony’ show, the students in this sample ranked “lecturing in an entertaining manner’ nearly at the bottom of the list (number 15). Both males and females ranked ‘indicating what is important for exams’ as number 5, again suggesting the importance of grades rather than learning. Data were categorized by sex of respondent to determine whether differences exist between female and male perceptions of important teacher characteristics. It can be seen from Table 6 that some differences exist but appear to be of little practical significance. Indeed, it may be suggested that the extent of agreement between the sexes in ranking the traits is unexpectedly high. The absence of large differences in perceptions of females and males concerning important characteristics of effective teaching may argue for those who suggest that both sexes want and expect the same things from the educational process. Table 7 shows the average rating assigned to each trait by females and by males. Two facts are immediately apparent from these ratings. First, the range of importance ratings is almost identical between the sexes, with a low of 4.7 in each case to a high of 6.4 for males and 6.6 for females. Secondly, it is worth noting that none of the trait’s average rating by either sex was below the mid-point on the scale. The average ratings by females and males are notably consistent. In only four cases does the difference between female and male ratings exceed .3 on the 7 point scale. Females rated emphasizing concepts, exhibiting concern for students and easy to outline lectures slightly higher than did the male students (a .4 difference in each case). Males rated entertaining lectures slightly higher (.5) than did the females. The fact that so few students rated any of the traits as being of little importance is interesting. Did they assume that since the directions stated that the traits were “commonly regarded as - characteristics of effective teaching’ they must all be of some importance? Or do they honestly consider all of the traits to be necessary for effective teaching? From the data it is impossible to say. However, there were no ties in weighted scores for the total sample (the closest being a 1 point difference between the eighth and ninth ranked items) which would seem to indicate that students did attempt to rank the items. On the other hand, if all students had rated an item as extremely important (7) its score would have been 1799, (7 times 257 students). If all had rated the item of no importance its score would have been 257, (1 times 257). The maximum difference then would have been 1542 points. There was actually a difference of only 472 points between the top rated item (with a score of 1678) and the lowest rated item (a score of 1206). Thus, students used only about the top one-third of the possible range of ratings. This would seem to indicate that all items were considered important. CONCLUSIONS The analysis of student ratings of the 17 teacher traits in this study indicates that male and female students tend to agree on which characteristics are important to teaching effectiveness. A possible problem is indicated by the apparent emphasis given by students to characteristics which impact directly on grades more than on learning. This analysis tends to corroborate Developments in Business Simulation & Experiential Exercises, Volume 8, 1981 135 the evidence found in other studies suggesting differences in perception between students and teachers (Stevens and Marquette, 1979). If teachers base their effectiveness standards on learning criteria and students base theirs on grading criteria the use of student evaluations may be inappropriate as a means of judging teacher effectiveness. In the present study differences do exist in student ratings of selected teacher traits (when examined on the basis of sex of student); however, these differences appear to be of little practical significance. (See Table 7). it should be noted that student respondents in the present study appear to have a very strong grade orientation. This orientation may influence the kinds of response generated. For example, it seems logical to ask if students with a strong learning orientation would respond similarly to those queried in the present study. There are a number of avenues available to future researchers in this area. The present study should be replicated and extended. Researchers may wish to gather data from a random sample of students across different majors and colleges within a particular university setting. In addition, results gathered from samples at different universities might provide further insight. The importance of key teaching behaviors or traits might be explored giving consideration to a variety of student attributes other than sex of student (e.g. , graduate vs. undergraduate standing, age, work experience, major). REFERENCES 1. Baba, V., and M. E. Ace. ‘Leader Behavior in the University Classroom.’ Western American Institute for Decision Sciences Proceedings, (1978), 184-186. 2. Banziger, G. J., and R. K. Smith. ‘Student Motivation and Course Evaluation.” Western American Institute for Decision Sciences Proceedings, (1978), 181-183. 3. Costin, F. , W. T. Greenough, and R. J. Menges. “Student Ratings of College Teaching: Reliability, Validity, and Usefulness,’ Review of Education Research, 41(1971), 511-535. 4. Cummings, L. L., and D. P. Schwab. Performance in Organizations: Determinants and Appraisal. Glen- view, IL: Scott, Foresman, 1973, 106. 5. Elmore, P. B., and K. A. LaPointe. “Effect of Teacher Sex, Student Sex and Teacher Warmth on the Evaluation of College Instructors.” Journal of Educational Psychology, 67(1975), 368-374. 6. Kinecki, A. , and C. A. Schriesheim. Teachers as Leaders: A Moderator Variable Approach.’ Journal of Educational Psychology, 70(1978), 928-935. 7. Lein, C. D., and M. Merz. ‘Faculty Evaluation in Schools of Business: The Impact of AACSB Accreditation on Promotion and Tenure Decisions. Collegiate News and Views, 31(1978), 2, 21-24. 8. Maier, N. R. F., L. R. Hoffman, J. J. Hoover, and W. Superior-Subordinate Communication: A Statistical Research Project. New York: American Management Association, 1961. 9. Miller, D. W. “Dangers of Using Student Evaluations for Administrative Purposes.” Collegiate News and Views, 31(1978), 3, 2-4. 10. Miller, D. W., S. Brokaw, and F. Shaaban. ‘Administrative Uses of Student Evaluation of College Instruction.” Annual Meeting of the American Institute for Decision Sciences Proceedings, (1977), 589. 11. Neeley, S. E. and B. F. Schaffer. Student Evaluations of Business Courses: Some Findings.” Journal of Business Education, 54 (May 1979), 366-368. 12. Peterson, R. A. , R. A. Kerin, and W. S. Martin. “Teaching Effectiveness in the Decision Sciences: The Use of Student Evaluations.” Decision Sciences, 9 (1978), 712-714. 13. Rodin, M., and B. Rodin. ‘Student Evaluation of Teachers.” Science. 177(1972), 1164-1166. 14. Schwab, D. P. “Attitudes Measured Prior to Taking a Course and Subsequent Evaluations of that Course and Instructor.” Annual Meeting of the Academy of Management Proceedings, (1977), 454. 15. Stevens, G. E. and R. P. Marquette. ‘Differing Student and Faculty Perceptions of Teaching Effectiveness and the Value of Student Evaluations.’ POD Quarterly, 1(Winter 1979), 207-219. 16. Sullivan, A. M., and C. R. Skanes. Validity of Student Evaluation of Teaching and the Characteristics of Successful Instructors. Journal of Educational Psychology, 66(1974), 584-590. 17. Swanson, R. G. Teacher Evaluation as a Function of Leadership Style: A Multiple Correlational Approach, Unpublished doctoral dissertation, North Texas State University, 1974. 18. Warren, W., and W. E. O’Connell. “The Effect of Instructor Reputation and Personality Upon Instructor Evaluations.’ Annual Meeting of the Academy of Management Proceedings, (1977), 45-49. 19. Wilson, D. , and K. 0. Doyle. Student Ratings of Instruction.” Journal of Higher Education, 47(1976), 465-470. 20. Witty, P. “An Analysis of the Personality Traits of the Effective Teacher.” Journal of Educational Research, 40(1947), 662-671. Developments in Business Simulation & Experiential Exercises, Volume 8, 1981 136 Table of Contents Volume 8, 1981 The Promotion: Human Sexuality in Organization The Simulation of Chaos Leadership Development in a Simulated Urban/Suburban (U/S) Environment Tomed: A Computer Game Emphasizing Social Responsibility/or/why the Pop-Top Can? Integrated Brain Activity and the Manager's Job: Utilizing the Troika Model What does R2 Have to do with a Product Management Course? An Analysis of the Effects of Jungian Problem-Solving Style Dimensions on Marketing Decisions Bargaining Behavior in Personal Selling and Buying Exchanges Extending the Simulation Product Life Cycle A Generalized Algorithm for Designing and Developing Business Simulations Operationalizing a Test of a Model of the Use of Simulation Games and Experiential Exercises Pygmalion and Perception: An Experiential Exercise Behavioral Consequences of Reward Regarding Employee Absenteeism in an Industrial Setting: An Operant Conditioning Approach The Simlab Program: The Use of Experimental Simulation and Process Analysis for the Development of Management and Organizations Designs for Research on Simulation-Games, Cases, and Other Experiential Exercises The Role of Students in The Case Method Weaknesses in Research Design Critical Variables in Research on the Educational Value of Management Games Research Questions for Cases Research on the Learning Effectiveness of Business Simulation Games - A Review of the State of the Science The Effects of Valuation Techniques on Holding Cost During Inflationary Periods: A Simulation Exercise The Operations Simulation - A Study in Game Development Applying Guided Design to the Production/Operations Management Course: A Progress Report and Evaluation Terminal Data Entry and Retrieval Systems Simulations and Microprocessors Microcomputers and Related Technology for Simulation Gaming Microcomputers - A New Technology for Innovations in Business Simulations Microprocessor Controlled Interactive Video Simulation Business Game Design: From Theory to Practice The Success of a Computerized Simulation in Microeconomic Pedagogy The Test Preview Game: Applying the Game Show Format Providing a Real World View of the Personal Function: A Simulation Finding an Effective Means of Teaching Managerial Behavioral Skills: Two Different Experiential Teaching Methods Compared A Management Development Program Based on the Experiential Learning Model Participant Type Differences in Response to Experiential Methods: An Informal Look Preparing Student Groups to Participate in Experiential Group Projects: An Organizational Development Approach An Instrument for the Assessment of Learning Dimensions: A Progress Report of the Learning Dimension Scale (LDS) An Empirical Analysis Relating the Learning Style Inventory to Memory and Logical Ability Teaching Styles in Simulation Experiential Learning Versus Traditional Teaching Styles Student Perceptions of Effective Teaching Behaviors Problems in Evaluation of Experiential Learning in Management Education Students' Perceptions of Learning by Simulation A Relative Evaluation of Experiential and Simulation Learning in Terms of Perceptions of Effected Changes in Students Overview of Computer Based Business Games in Business Policy Classes Behavioral Decision Theory and Business Policy Giving Accounting Students Writing Experience as Job Preparation Getting to First Base with MBO: An exercise for Writing and Evaluating Objectives Dimensions of Conflict in Experiential Learning Consumer Alienation and Perceived Relevance of the Business Simulation Using the Self-Reference Criterion to Simulate Culture in Internationalized Business Course Experiential Learning in a Cross Cultural Setting- The Practice of Simulation Approach to Business Education in Japanese Universities Student' Perceptions of the Use of a Computerized Simulation in Teaching Management Information Systems Using the Case Study Approach to Develop a Microcomputer Bases, Fully Integrated, Data Base Driven, Management Information System The Case Study as a Tool for Organizational Change: Applying the Steel Ax to the Designers of Management Information Systems Toward a Theory of Teaching Business Policy A Case Study in the Use of Experiential Learning (A Management Game Simulation) to Enhance Student Understanding of Strategy Evaluation and Policy Formulation Suggestions for Integration of the Business Administration Core Publishing Opportunities and Requirements for Business Simulation and Experiential Learning Materials How do we Apply Experiential Learning Intercollegiate Case Competitions for M.B.A. Students: Initiation and Implementation Teaching Business Policy and Strategy Using the Incident Process The Learning Co-Op Approach to the Core Policy Course Meeting the Managerial Skill Shortage - Is Academia Up to It? An Empirical Analysis of Experiential Learning Reinforcement International Experiential Learning: Experience is the Best Teacher Encouraging Student Participation During International Academic Programs European Summer Study Program: Can you, Should you, What Does it Take? Travel Seminars in Europe: How can I Direct One? International Experiential Learning: Student Evaluation ABSEL: Empirical Findings on the State of the Association Sensitivity of Performance Scores in Business Simulations Improving the Learning of a Business Simulation Game by Increasing the Process Content Student Participation in Deciding Performance Criteria for Grading in an OB Course: An Exercise and a Case Study Markup for Profit: A Simulated Self-Administered Experience in Retail Pricing The Investment Decision Game: An Experiential Learning Approach to Stock Market Decisions Through Gaming CHIPS: A Marketing Channels management Game External Validation: An Experimental Approach to Determining the Worth of Simulation Games Teaching Performance Appraisal Skills: An Experiential Approach In Support of Experiential learning: Results of a Follow-Up Survey The Introductory Management Course: Taking Theory Application One Step Further Decision Efficiency and Effectiveness in a Business Simulation The Implications of Cognitive Processing Variables and the Complex Decision Simulating the Simulation for Enhanced Player Rationality Simulation/Experiential Learning Audit