AN EVALUATION OF THE MINITAB PACKAGE IN TEACHING BUSINESS STATISTICS CONCEPTS Developments in Business Simulation & Experiential Learning, Volume 11, 1984 111 AN EVALUATION OF THE MINITAB PACKAGE IN TEACHING BUSINESS STATISTICS CONCEPTS Marcelline R. Fusilier, Creighton University Pamela Hammers Specht, University of Nebraska-Omaha Daniel C. Ganster, University of Nebraska-Lincoln Conrad Jackson, Tulsa University ABSTRACT Two studies empirically evaluated the effectiveness of computer assisted instruction in facilitating students’ comprehension of the concepts of the Central Limit Theorem and Type I and Type II error. Undergraduate students were assigned to experimental and control groups. The experimental group’s regular class material was supplemented with a computer assisted instructional exercise from the Minitab Package (Ryan, Joiner & Ryan, 1976). The control group did not experience the exercise. Results indicated that an instructional program supplemented with Minitab was as effective as a program utilizing only traditional instructional methods. The potential utility of the Minitab package in business education is discussed. In recent years, the use of computer-assisted instruction (CAI) techniques has proliferated in educational institutions. These techniques have been applied to the teaching of subjects as widely varied as mathematics and foreign languages. Furthermore, they are used at all levels of education, from early elementary to postgraduate training. The fast implementation of the CAI systems, however, has often preceded a critical examination of the utility of the computer packages in promoting learning of the various subjects for which they are used. This is becoming an issue of widespread concern as educational institutions are investing time and money in implementing computer assisted instructional packages that have not been evaluated for their utility in promoting comprehension of subject matter (Williams & McDonald, 1982). Business education, like many other areas of higher education, has been involved in this proliferation of CAI. Many computer- assisted instructional packages are touted as helping students to more effectively comprehend business concepts (Milkovich & Mahoney, 1975; Robson, 1975) but few have been systematically and rigorously evaluated in this respect. EVALUATION RESEARCH CONCERNING CAI A review of the literature reveals a variety of empirical studies concerning the effects of computerized instruction. For example, in a large study involving several different subject populations, Suppes and Morningstar (1969) compared the academic performance of students who participated in CAI programs versus those who learned without computer assistance. Three different groups of subjects were studied, (a) elementary school students in California, (b) elementary school students in Mississippi, and (c) college-level students enrolled in a Russian language course. Among the elementary school students, the researchers were concerned with the use of CAI in mathematics courses. The dependent variable of academic performance was operationalized as each student’s score on the arithmetic portion of the Stanford Achievement Test. Two evaluations of the CAI math program in California took place, each in a different year and involving different students. With regard to the first year, seven different math classes were involved in the study. The classes represented four elementary school grades. In three out of seven classes, the CAI group performed better at a statistically significant level (p < .05) on the Stanford Achievement Test than did the control group. In the second year, the experimental group performed significantly better than the control group in three out of six classes. The results of the evaluation conducted in Mississippi schools were less ambiguous than those of the California schools. The experimental (CAI) group performed better on the Achievement Test than the control group in all six of the classes studied. Similarly, testing in Russian classes at Stanford University indicated that students using CAI performed better on class examinations than did those not using CAI. In a later study, Roecks and Chapin (1977) found no statistically significant differences in mathematical achievement test gain scores between elementary school students who had experienced CAI and those who had not. Thompson (1977) on the other hand, found more positive results when the role of CAI was examined in a college- level macroeconomics course. The findings indicated that students whose instruction was supplemented with a computer package performed 10% better on exams than those not exposed to the computer package. More recently, an extensive empirical study of the effectiveness of CAI was undertaken by the Ontario Institute for Studies in Education (Gershman & Sakamoto, 1981) during the two year period from May 1978 to April 1980. The project was designated as CARE (Computer- Assisted Remediation and Evaluation). Its purpose was to create and evaluate CAI sequences for Intermediate mathematics courses offered in Ontario’s secondary schools. To measure achievement gains attributable to CARE, pretests were administered to students in Grades 7-10 at eight participating schools. Pretest results indicated no significant difference between the math scores of the CARE group and the comparison group. Posttest results indicated however that CARE students not only improved significantly from pretest to posttest, but also improved significantly more than non- CARE students at the same school (t = 4.9, p < .001). In a comprehensive analysis of 40 published empirical studies involving computer assisted mathematics instruction, Burns and Bozeman (1981) examined and synthesized research findings relative to the effectiveness of such instruction in elementary and secondary schools. These authors used meta-analysis, which employs a common measure of treatment effectiveness, effect size, to obtain a quantitative synthesis of research outcomes. Burns and Bozeman defined effect Developments in Business Simulation & Experiential Learning, Volume 11, 1984 112 size as the mean difference on the outcome variable (student achievement) between treatment and control subjects divided by the within group standard deviation. Their results indicated a significant enhancement of mathematical achievement in those programs supplemented by CAI. Specifically, the research findings suggested that (a) mathematics instructional programs supplemented with CAI were significantly more effective in fostering student achievement than programs not utilizing CAI, (b) CAI drill/practice programs were significantly more effective in promoting increased student achievement at both the elementary and secondary levels, and (c) supplementary CAI drill/practice programs were significantly more effective in stimulating achievement gains among boys at the intermediate grade level than non- CAI programs. This final relationship was not demonstrated for intermediate level girls. With regard to business education, Mahoney and Milkovich (1975), among others, have noted the intuitive appeal of CAI for management training. Casual observations of management skill acquisition through CAI have been reported but no investigations have yet been conducted to test this contention. This neglect is surprising given the increasing reliance on computer simulations and other CAI techniques in colleges of business throughout the United States. On the basis of the present review of the literature, it is clear that CAI has a largely positive effect on the learning of mathematics for primary and secondary school students. Furthermore, Suppes and Morningstar (1969) and Thompson (1977) found positive results regarding the use of CAI in college courses. The generalizability of these findings to business education, however, is highly tentative. In order to draw conclusions pertaining to the effectiveness of CAI for teaching business concepts, evaluation research germaine to this particular area is needed. HYPOTHESES The purpose of the present research, then, is to investigate the utility of CAI with respect to a particular aspect of business education, the teaching of business statistics. The computer package employed here is Minitab (Ryan, Joiner & Ryan, 1976) a widely used computer aid in teaching applied statistics. Although the effectiveness of this package has been discussed in a tutorial paper (Ryan, 1975) and a symposium, (Berenson, 1982). There is no published evidence, of which the present authors are aware, concerning its effect on facilitating learning of statistical concepts. Hence, two studies were conducted to investigate the effect of the use of Minitab on students’ acquisition of two statistical concepts, the Central Limit Theorem and Type I and Type II error. The two topics, Central Limit Theorem and Type I and Type II error were chosen because all the statistics instructors at the institution at which these studies were conducted reported that on the basis of past teaching experience, these two concepts were among the most difficult for business students to grasp. This may be the case because students cannot easily work through examples of either concept by hand as they can, say with the computation of a standard deviation. Specifically, a computational example of the central limit theorem involves drawing samples of a given size from a population, computing the samples means, then plotting the sampling distribution of the mean. Furthermore, in order to truly get an example of the central limit theorem, a student would have to plot sampling distributions for means computed from samples of different sizes. Similarly, a computational example of Type I and Type II error commission would involve drawing samples from a population, computing a statistic, conducting a test of statistical significance, comparing the test statistic to some critical value set at a pre-established significance level, then rejecting or not rejecting the null hypothesis on the basis of that comparison. The veracity of the test could then be assessed. These processes are, of course, extremely laborious for the student when done by hand or with a small calculator. Minitab can randomly generate data samples of a given size and either plot the distribution of the sample means or conduct tests of statistical significance in seconds. The Minitab package therefore comprises a time and labor saving instructional tool, providing the student with examples that he/she would otherwise probably not be able to experience. Hence, it seems reasonable that Minitab exercises should provide students with a better grasp of difficult statistical concepts than they would have if Minitab were not utilized. This idea was tested in the present set of studies. Specifically, in the first study, it was hypothesized that students who experienced a Minitab exercise concerning the Central Limit Theorem would perform better on exam items concerning that topic than students who were not involved with the exercise. With regard to the second study, students experiencing a Minitab exercise concerning Type I and Type II error were predicted to perform better on exam questions regarding that topic than students who were not exposed to the exercise. METHOD Study I Subjects. Subjects were 86 undergraduate students enrolled in an introductory business statistics course at a medium-size private university in the midwestern United States. Four sections of the course, taught by two different instructors, were involved in the study. Two of the sections met during the evening and two met during the day. Each instructor taught one day and one evening class. Data collection took place during the fall semester, 1982. Treatment. All students were exposed to the Minitab package in their statistics course. All were familiar with Minitab’s format and had used it for various statistical exercises unrelated to the central limit theorem prior to data collection. At the time of data collection, the experimental group was assigned an exercise that illustrated the central limit theorem while the control group worked through a different exercise concerning another topic. All sections were given the same in-class lecture concerning the central limit theorem. The exercise (Ryan, Joiner & Ryan, 1976, 6-9, 6-10, pp. 113, 115) involved two parts. In the first, students were required to generate the following numbers of randomly drawn samples of 100 observations from a normally distributed population: (a) 1, (b) 2, (c) 6, (d) 10, and (e) 16. The computer generated the data. Histograms of the sample means for each of the five sets of samples were then plotted. The second part of the exercise was identical to the first except that the samples were drawn from a non-normally distributed population. The purpose of this assignment was to demonstrate to students that as the number of samples drawn increases, the sampling distribution of the mean centers around the population mean and tends toward normality even if the population is not normally distributed. Developments in Business Simulation & Experiential Learning, Volume 11, 1984 113 Students were counterbalanced across treatment and control groups on the basis of the section in which they were enrolled and their instructor. That is, one instructor’s day class and the other instructor’s evening class constituted the experimental group. The remaining students, enrolled in the first instructor’s evening section and second instructor’s day section, formed the control group. Entire sections were assigned to groups so as the minimize communication between, and contamination of, the treatment and control groups. The Minitab assignment was graded as correct if the student accurately completed both parts and provided a written interpretation of the information contained in the printout. Only students who had completed the assignment correctly were included in the experimental group. Those who did not complete the assignment were dropped from the study. Dependent Variable Measures. The dependent variable was comprehension of the Central Limit Theorem. Two questions were included on one of the four class examinations in order to measure this criterion. The exam was given one week after students were required to complete their Minitab assignment. These questions were embedded among a number of others so as not to make their significance to the research obvious. One question was a short essay: “In your own words, state the Central Limit Theorem." The other question required a true or false short answer. It was: “The standard deviation computed form samples of size n is a random variable. Briefly explain your reasoning." The questions were coded as correct if the response was consistent with the central limit theorem. With regard to the first question, a correct response was defined as including all parts of the central limit theorem. Study II Subjects. Subjects were 70 undergraduate students enrolled in the same introductory business statistics course as those in Study I. Data for Study II were collected in the spring semester, 1983. Two sections of the course were included in the study, one section serving as the experimental group, the other as the control. Both were taught by the same instructor. Treatment. As in Study I, all students were familiar- zed with the Minitab package prior to the time of data collection. At data collection time, the experimental group was assigned two exercises concerning hypothesis testing (Ryan, Joiner & Ryan, 1976, 1976, 7-7, 7-8, p. 131) while the control group received an alternate exercise not involving hypothesis tests. In the first experimental exercise, students were required to generate 20 randomly drawn samples of 16 observations from a normally distributed population with p = 64. The null hypothesis to be tested was H0: p = 64 versus the alternative that H1: p 64. The computer generated the data, then calculated z-tests for each of the twenty samples generated. The second exercise was identical to the first except that the samples were drawn from a population where p = 63. In both exercises, students tested the null hypothesis at the .05 and .10 significance levels. They then compared the number of times an Incorrect decision was reached in this case, rejection of H0 to the number of times out of 20, on average, they would have expected to make an incorrect decision. Assignments were graded as in the first study. Only those students who completed their exercises correctly were included as subjects in the study. Dependent Variable Measures. The dependent variable was comprehension of Type I and Type II error. It was measured with three questions which appeared on one of the four class examinations. The exam was given a week after students completed their Minitab assignment. One of the three questions was a short essay: “Based on personal experience, a manager believes that p = 15 for a certain population. An article in a business publication reports that a two- tailed test of the null hypothesis that í = 15 was not rejected at the .05 significance level. Assume proper statistical procedures were used. Is 15 the true value for the population? Explain your reasoning.” The other two questions required a true or false answer. They were: (a) “In a test of a hypothesis using a .05 significance level, there is a 5% chance that the null hypothesis is wrong.” and (b) “It is possible to conduct five independent tests of hypotheses using proper statistical procedures and have the result be incorrect in each case (that either Type I or Type II error is committed each time).” The questions were coded as correct if the response was consistent with the concepts of Type I and Type II error. RESULTS Study I Data were analyzed using z-tests of the differences between proportions of correct responses across the experimental and control groups. The experimental group contained 48 observations and the control group, 38. For the essay question, the difference between the groups was not statistically significant at a traditionally accepted level (z=1.296, n. s.). The results concerning the true-false questions, however, did suggest a statistically significant difference (z=L636, p. < .05). Study II Data were analyzed using the same type of z-test as in the first study. Here, the experimental group contained 39 observations and the control group, 31. Results were uniformly nonsignificant for all of the questions (essay, z = .726; first true-false, z .183; second true-false, z = .986; all n. s.). DISCUSSION One of the studies reported here evidence marginal support for the hypothesis that CAI in the form of Minitab would facilitate students’ learning of business statistics concepts. With regard to the central Limit Theorem, involvement in a Minitab exercise did seem to contribute positively to exam performance. No effect on exam performance was found when learning of Type I and Type II error was tested. Thus it appears that Minitab does have utility in facilitating students’ grasp of at least one statistical concept. The support, however, is not over whelming. This, of course, does not mean that Minitab lacks importance as an instructional tool in business statistics courses. It may serve to facilitate learning of other statistical topics beside the central limit theorem, or make students feel more comfortable about interacting with a large computing system. The present findings suggest that the reasons for implementing CAI in business statistics or management courses might be scrutinized more carefully than they have been in the past. If the goals of a course include increasing the students’ familiarity in interacting with a computing system, CAI may be the answer. On the other hand, if CAI Developments in Business Simulation & Experiential Learning, Volume 11, 1984 114 is used to promote conceptual learning beyond what can be achieved using traditional teaching methods, the students and the instructor may not get the strongly positive results that are desired. Whether or not these results make a definite statement relative to the effectiveness of CAI as an instructional medium remains open to question. Magidson (1978) has summarized the literature concerning CAI by noting that an instructional program supplemented with CAI is at least as effective, and often more so, than a program utilizing only traditional instructional methods. The present results are consistent with that statement. In no case did the experimental group perform at a lower level on their exams than the control group. A possible alternative explanation for the nonsupportive findings reported here might concern the degree to which the Minitab package was integrated with the course as a whole. Jones and O’Shea (1982) point out that students often perceive CAI exercises as being “tacked on” to a course rather than as an integral part and thus fail to give it sufficient emphasis. If this were the case in the classes tested, it is not surprising that Minitab had less influence on exam performance than had been anticipated by the researchers. On the other hand, however, it may be that CAI-supplemented instructional programs are no different in effecting student comprehension than traditional programs and that the present results are reflective of that assumption. As Lawton and Gershner (1982) note, few researchers are willing to guarantee that students will or will not learn through the use of CAI. Hence, the wisdom of using CAI as a teaching method in college- level business courses is still questionable. The frenzied adoption and uncritical acceptance of computers as a teaching tool (Williams & McDonald, 1982) may not provide the much sought-after increases in learning that were predicted five to ten years ago. Only more evaluation research, employing different CAI methods, and testing across various types of business courses, will provide an answer to this pressing problem. REFERENCES (1) Berenson, M. K. Using Minitab on classroom survey data: The “Ultimate” approach. Tutorial workshop presented at the annual meeting in the American Institute for the Decision Sciences, San Francisco, CA, November 22-24, 1982. (2) Burns, P. K. & Bozeman, W. C. Computer-assisted instruction and mathematics achievement: Is there a relationship? Educational Technology, 1981, 21, 32-39. (3) Gershman, J. & Sakamoto, E. Computer-assisted remediation and evaluation: A CAI project for Ontario Secondary Schools. Educational Technology, 1981, 21 40-43. (4) Jones, A. & O’Shea, T. Barriers to the use of computer assisted learning. British Journal of Educational Technology, 1982, 13, 207-217. (5) Lawton, J. & Gershner, V. A review of the literature on attitudes towards computers and computerized instruction. Journal of Research and Development in Education, 1982, 16, 50-55. (6) Magidson, E. M. Issue overview: Trends in Computer-assisted instruction. Educational Technology, 1978, 18, 5-8. (7) Milkovich, C. T. & Mahoney, T. A. Computer simulation: A training tool for manpower managers. Personnel Journal, 1975, 54, 609-612, 637. (8) Robson, W. Is it true what they say about videodiscs? Training, 1975, 12, 80-82. (9) Roecks, A. L. & Chapin, J. Mathematics achievement in schools having and not having CMI: An assessment of MICA. Paper presented at the annual meeting of the American Educational Research Association, New York, New York, April 4-8, 1977. (10) Ryan, T. A. Jr., Joiner, B. L. & Ryan, B. F. Minitab Student Handbook, North Schituate, Mass.: Duxbury Press, 1976. (11) Ryan, T. A. Jr. et al. Teaching statistics with Minitab II. Paper presented the Conference on Computers in the Undergraduate Curricula, Fort Worth, TX, June 16-18, 1975. (12) Suppes, P. & Morningstar, M. Computer-assisted instruction: Two computer-assisted instruction programs are evaluated. Science, 1969, 166, 343- 350. (13) Thompson, F. A. Teaching information processing systems implementation at Riverside City College, 1976-77; An experiment in educational innovation. Unpublished paper, Riverside City College, Riverside, CA 1977. (14) Williams, D. A. & McDonald, D. H. The great computer frenzy. Newsweek, 1982, December 27, 68. Table of Contents Volume 11, 1984 Simulation Gaming as a Means of Researching Substantive Issues: Another Look A Further Test of the Group Formation and its Impacts in a Simulated Business Environment Impact of Economic Patterns on Student Performance in Computer Business Simulation Games Majority Fallacy Game with Independent Student Simulation and a Case Introducing the Marketing Channel Laboratory A Comparative Evaluation of a Marketing Game A Study of Comparative Effectiveness of Problem-Solving Technologies The Impact of Hierarchical and Egalitarian Organization Structure on Group Decision Making and Attitudes Risk-Free Decision Making The EX-STRA Export Strategy Game Computer Education for Management Students Developing a Computer Game/Job Simulation to Teach Functional Literacy Skills Experiencing Socialization First Hand: An Experiential Exercise in Organizational Socialization Networking Distributive Versus Integrative Approaches to Negotiation: Experiential learning Through a Negotiation Simulation Managerial Education and the Real World: Foudations for Designing Educational Tools Diagnosing Group Climate to Improve Supervisory Effectiveness Student background as a Factor in Simulation Outcomes: The Collective bargaining Example The Use of Pre-Plays in Management Education Experiencing the Process Debrief: A Workshop ABSEL Megatrend Roots MEGATRENDS for Business Simulation and Experiential Learning The Effects and Consequences of the Megatrends on Simulation Gaming: One View Opportunities for the Future: ABSEL's Role Experiential Learning-Based Discussion vs. Lecture Based Discussion: A Comparative Analysis in a Classroom Setting An Evaluation of the Minitab Package in Teaching Business Statistics Concepts A Path Analytic Study of the Effects of Alternative Pedagogies Developing and Using Weighted Application Blanks: An Experiential Exercise Building Airplanes Individual vs. Group Grade: An Exercise in Decision making A Marketing Plan Exercise: Development of Interteam Cooperation Using a Coordinated Experiential Approach Using Student Experience as the Basis for a Consumer Behavior Learning Exercise Student Evaluations of Instructors: What do Students Believe? A Description of the SOFTCAT Computer Assisted Teaching System Comparisons of Practitioners' and Professors' Perceptions of Business Policy Content and Learning Methods The Perceived Relationship Between Pedagogies and Attaining Course Objectives in the Business Policy Course The Use of Simulation in the Teaching of Business Policy A Research Study on Strategic Decisions in a Business Simulation Strategic Management Decision Making Researched Via Simulation Gaming Using Simulation to Investigate Factors in Competitive Bidding Combining Experiential Learning and management Assistance A Model for Teaching Management Skills Putting Experience Back into Experiential Learning: A Demonstration The Teaching and Behavioral Measurement of Managerial/Organizational Competencies: Developing Experiential Exercises and Simulations A Simulation Game Model for Conglomerates QCLAB - A Microcomputer Laboratory in Quality Control CTSS: A Commodity Trading Simulation System Problem Solving: An Exercise on Learning, Coaching, and Operant Conditioning A Demonstration of the Effects of Feedback as a Category of Reinforcement The Assessment of Feedback and Disclosure in Interpersonal Relations: An Experiential Exercise A Study to Determine Whether the Teaching of Basic Grammar Skills in Business Communication Classes Improves Students' Business Letter Writing Corporate Maladies Through the Eyes of the Memo Writer: A Seldom Used Experiential Tool Executive Bailout at Shake & Spear, Inc. The H.E./L&P Merger Intercultural Nonverbal Communications: An Experiential Exercise The Evolving Business Policies Course - Is Management Gaming the Logical Pedagogy? The Use of Decision Simulations in Management Training Programs: Current Perspectives Humanizing the Business of Medicine: The Use of Simulated Patients to Train medical Students Systematic Integration of Simulation Methods in a Graduate Management Curriculum Modeling Non-Price Factors in the Demand Functions of Computerized Business Using Spacial Relationships to Estimate Demand in Business Simulations Two Algorithms For Redistribution Of Stockouts In Computerized Business Simulations Leadership And Strategic Behavior A Comparison Of Two Business Strategy Simulations For Microcomputers Incorporating Decision Support Systems Into Management Simulation Games: A Model And Methodology Using Micro-Computers To Support The Analysis Of Complex Cases: It's As Easy As 1-2-3 Strategic Formulation Consistent With Pims: A Micro-Computer Application