CHANGING PERCEPTIONS OF LEARNING IN A SIMULATED ENVIRONMENT New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 101 CHANGING PERCEPTIONS OF LEARNING IN A SIMULATED ENVIRONMENT David J. Fritzsche Rochester Institute of Technology Measures of student attitudes and perceptions have been used in numerous studies attempting to measure learning in a gaming environment (1,2,3,4,5). These have ranged from statements of the most valuable aspect of the game to attitudes toward gaming as a pedagogical technique. However, the writer is not aware of any attempt to measure students’ perceptions of the contribution experiential techniques make toward their acquisition of business knowledge. This paper reports an attempt to construct an instrument designed to measure students’ perceptions of their acquisition of knowledge of business. Such an instrument can be put to several uses in the study of experiential learning processes. First it could be used to compare experiential techniques with nonexperiential teaching techniques. Second it could be used to evaluate revisions to experiential learning environments. Third it could be used to evaluate alternative experiential teaching techniques. Constriction of the instrument began with developing a set of factors which represents various aspects of business knowledge. The following list of six factors was selected: 1. business concepts 2. business problems 3. committee work 4. relationships among operating variables 5. business experience 6. analytical techniques Each of these factors was incorporated into a declarative sentence concerning learning, and students were asked to respond to each statement using a Likert type 5 point scale containing “strongly agree” and “strongly disagree” polar adjectives plus a “don’t know” option. (see Appendix for instrument). A second set of statements incorporating the same factors using different words was developed to check for consistency of response. While some of these statements include the word “marketing”, in most cases simply replaing “marketing” with another discipline name should provide the appropriate adoption. Thus the instrument consisted of a short 12 question series which could be completed by the student within five minutes. The perception of learning instrument was administered to 84 student; at the Rochester Institute of Technology who were enrolled in tie principles of marketing course. These students were generally juniors and sophomores without an extensive number of previous business courses. The students were asked to complete the instrument just prior to the midterm exam and again at the end of the course before the final exam. New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 102 FINDINGS The findings will be analyzed in two parts. First we shall examine the reliability of the instrument. Then we shall look at the changes in student perceptions over the period of the course. Analysis of Reliability Reliability was tested by comparing the response to each pair of statements relating to each factor. For example, the following statements were used to elicit students’ perceptions of their learning concerning business problems: 1. I am learning important business concepts. 2. I am discovering important marketing principles. IMAGE Table 1 The response to the pair of statements associated with each factor was analyzed using the Spearman rank correlation procedure. The resulting significant rs values are shown in Table 1 for each factor for both the midterm and the end of course assessments. Four of the factors yielded higher rs values for the end of course evaluation. Only one of the factors, business concepts, yielded a consistent rs value of over .6. Thus while the relationships are all significant, few of them are really as strong as one would desire. The rs values could have resulted from responses which were generally not more than 1 scale unit different. Thus a closer examination of each of the factors was made by looking at the magnitude of the difference in student response to each of the statement pairs. If the difference in response by a student to a pair of statements was one unit off, the difference would not be deemed too critical. However, if the difference was two or three units, the statements are clearly unreliable assuming they are both measuring the same factor. The data in Table 2 indicate the proportion of the total response which differed by no more than one unit of measurement. Three New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 103 of the factors yielded more than ten percent of the responses which were more than one unit different. Two of these factors, business experience and analytical techniques, yield less than ten percent at the end of the course. IMAGE Table 2 The numbers in parentheses in Table 2 show the proportion of the responses which were identical. The one unit differences were distributed approximately equally, both positively and negatively, around the no difference response for all factors with the exception of committee work and analytical techniques. The response differences for these two factors were overrepresented in either the positive or negative directions. The results of the Spearman rank correlation and the pair response difference analyses indicate that the reliability of the statement pairs is adequate for all of the factors except committee work and possibly analytical techniques. A second test of reliability was conducted by summing the scores of the first set of statements concerning the six factors, summing the scores of the second set of statements, and then measuring the relationship between the two sets of sums using the Pearson correlation techniques. The rationale here is that while there may be some differences in the response to individual statements, these differences may average out and provide an underlying consistency to the instrument. The correlation coefficient obtained from the data collected at midterm was .80, significant at the .001 level. The data collected at the end of the course yielded an r value of .91, again significant at the .001 level. While the former figure leaves something to be desired, the latter value is quite gratifying. The reason for the relatively large difference between the New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 104 two r values is not obvious. One explanation might be that at the end of the course the students more thoroughly understood the meaning of the statements and thus were more consistent in their responses. However, there is no empirical support for that interpretation. The analysis of the instrument indicates that on the whole it may be sufficiently reliable use as a research tool. The following section of this paper will report an initial attempt to use the instrument to measure changes in students’ perceptions of learning over time in an experientially based course. Analysis of Changing Perceptions Changes in students’ perceptions of learning were examined by comparing the response to each statement made at the midterm with the response to the same statement made at the end of the course. The mean response to each question is shown in Table 3. The means are tested for significant differences using the t test with the significance levels shown in the table. Significant positive changes are indicated for pairs of statements relating to the variables business concepts, relationships among variables, and analytical techniques. The latter variable is interesting in that it yielded poor results in the correlation test for reliability. It also ranked relatively low on the midterm statement pair response comparison. The writer suspects that the statements could be measuring two different factors and the students are perceiving an increase in learning with respect to each factor. New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 105 IMAGE Table 3 Number 1 refers to the first statement of the pair dealing with the variable and 2 refers to the second statement. One statement of the pair yielded significant positive change for the factors business problems, committee work and business experience. Committee work yielded the worst results to the reliability tests. The statement that was found to be significant referred to “discovering how to motivate people to get a job done.” The nonsignificant statement read “I am learning how to work in a committee environment.” It may very well be that students see these two factors as different and believed they were learning how to motivate people but were not learning how to work in a committee environment. The business problems statement “I am discovering many of the problems faced by ‘real world’ businessmen” yielded a significant increase. However, the second statement “Difficult decisions confronting the businessman are taking on real meaning” yielded an insignificant change. Apparently those two statements did not measure quite the same factors even though the statement pair response differences were quite acceptable in the reliability check. New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 106 The reliability of the business experience variable fell in the lower half of the group of variables. The statement “I am gaining valuable management experience which will be useful in my career” yielded nonsignificant results. The statement “Management responsibilities are becoming very real to me” yielded results which were significant at the .04 level. Again, the statements must have been measuring different factors. COMMENTS The instrument appears to yield mixed results. While the reliability tests indicated that most of the statements should possess adequate reliability, when the instrument was put to use, a number of the statement pairs appeared to be measuring different factors. Revisions are in order. The instrument did yield some interesting results. Students perceive that they are learning business concepts in the experiential environment. They also believe that they are discovering relationships among variables and are learning how to apply analytical techniques to business problems. Students believe they are learning how to motivate people, are discovering problems faced by the businessman and are learning about management responsibilities. The writer believes that the discovery of variable relationships, of problems faced by businessmen, and of management responsibilities are factors which experiential techniques have the unique capability of providing. With a few refinements the present instrument will enable one to make comparisons of this nature between experiential and nonexperiential teaching techniques and between alternate experiential techniques. Refinements are now in process. New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 107 New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 108 New Horizons in Simulation Games and Experiential Learning, Volume 4, 1977 109 REFERENCES 1. Dill, W. R., and N. Doppelt, “The Acquisition of Experience in a Complex Management Game,” Management Science, Vol. 10 (October, 1963), pp. 30-46. 2. Eliason, A., “A Study of the Effects of Quantitative Training,” Academy of Management, Vol. 15 (June, 1972), pp. 147-158. 3. McKenny, J. L. and W. R. Dill, “Influences on Learning in Simulation Games,” American Behavioral Scientist, Vol. 10 (October, 1966), pp. 28-32. 4. Roberts, Ralph N. and Steven E. Field, “Using Student Opinions in Evaluating Results With A Business Game,” Simulation Games and Experiential Learning in Action, The Proceedings of the Second National ABSEL Conference, Bloomington, Indiana, April 10-12, 1975, pp. 92-97. 5. Strother, G. B., A. C. Johnson, and H. E. Thompson, Educational Applications of Management Games, Office of Education, U.S. Department of Health, Education and Welfare, Washington, D.C., 1966. Table of Contents Volume 4, 1977 Double Play for Gaming Effectiveness Adaptive Rule Changes in Computer Simulation Gaming Œ A Means of Pedagogical Reactive Interchange Monte Carlo Simulation in Personnel Management Training Teaching About the Implementation of Job Redesign Using Simulation and Group Discussions An Interactive Simulation of Private Sector Collective Bargaining Leadership Evaluation and training through Behavioral Simulations: Method, Results and Future An assessment of the Effect of Experiential, Simulation and Discussion Pedagologies Used in Laboratory Sections of an Introductory Management Course An Experiential Understanding of the Trust Dimension Using Consulting Cases to teach Business Policy An Experimental Testing of Teaching Methodologies in Marketing Interpersonal Skill Development: The Experiential Training Unit (ETU) and Transfer of Training An Analysis of the Relationship between Personality characteristics and Preferred Styles of Learning Analysis of Effective Communication Skill Development in Graduate Business and Engineering Experiential Education Changing Perceptions of Learning in a Simulated Environment Student Perceptions: Simulation and the Corporate Policy Course Degree of Uniformity in Achievement Motivation Levels of Team Member: Its Effect on Team Performance in a Simulation Game Channel Conflict, Cooperation and Control: an Experiential Learning Exercise Differences in Experiential and Non-Experiential Learners' Reactions to Conflict between Individual and Organizations Behavior The Evolution and Evaluation of a Required, Senior-Level Course in Experiential Business Applications Building Management Skills through Problem Solving A Live-Case Approach to the Business and Society Course Experiential Learning: Toward the Development of a Theoretical Base and the Identification of Variables and the Hypotheses to Guide Research The Role of the Administrator in Experiential Learning and Simulations Some Thoughts on a Theory of the Use of Games and Experiential Exercises Three Applications of the Management of Learning Grid An Analysis of ABSEL: Its Past Achievements and Future Prospects New Horizons in Simulation Research Prediction of Academic Achievement in a Simulation Mode via Personality Constructs Sex Differences in a Bargaining Simulation COM-GAME: A Commodity Trading Game for Use in an Introductory Business Statistics Course A Financial Institution Management Game with Direct Participant Interactions A Non-Computerized Marketing Planning and Strategy Game Delphi in the Classroom: A Demonstration of Forecasting Economic Activity The Potential of Programmable Calculators for Processing Small Business Simulations Can a Small Predominantly Clack University Incorporate the Computer Simulation Gaming Teaching Methodology into it Curriculum Measuring the Effect of an Experiential Exercise Experiential Learning - Analysis of a Partial Success Predicting Participants' Performance and Reactions in an Experiential Learning Setting: An Empirical Investigation A Simplified, Non-Computerized Marketing Channels Game Manufacturers and Retailers: A Negotiation Game for Beginning Management Students Petroleum Management Game A Securities Dealer Simulator SIM ECO SOC with Business Curriculum Modules: A Simulation for Business Ethics and Morals The Picnic: A Perceptual Errors Exercise Salt III; an Experiential Exercise to Highlight the Interpersonal Dynamics Involved in the Negotiation Process Experiential Exercise on Values, Attitudes and Conflict Resolution in Organizational Behavior Kick'N Go: A Product Management and Social Responsibility Dilemma The Use of Self-Assessment Work-shops in a school of Business Administration The Dilemma of Self-Perception