AN EXPLORATORY STUDY OF THE EFFICACY OF SERVSAFE® ONLINE Developments in Business Simulation and Experiential Learning, Volume 34, 2007 178 AN EXPLORATORY STUDY OF THE EFFICACY OF SERVSAFE® ONLINE Andrew Hale Feinstein University of Nevada Las Vegas andy.feinstein@unlv.edu Michael C. Dalbor University of Nevada Las Vegas Michael.dalbor@unlv.edu Amy McManus University of Nevada Las Vegas mcmanus4@unlv.nevada.edu ABSTRACT ServSafe® is the most widely utilized food safety training course in the United States, providing certification to over 300,000 learners in 2005. Because of its prevalence, there has been interest in offering this training online. Further, as online programs continue to proliferate in the hospitality industry, the authors hope to provide insight into methodologies designed to evaluate the effectiveness of online instructional systems. The primary objective of this study is to determine whether there is a significant increase in learners’ food safety knowledge as a result of taking ServSafe Online. Three hundred forty-three participants with various backgrounds were used to take ServSafe Online. The mean improvement in the post-test score when compared to the pre-test score was more than 22 points. Overall, 81 percent of the respondents passed the exam, as compared to the 79 percent for those who took the traditional exam during 2005. A general linear model and analysis of variance was conducted to determine if various factors such as age and experience, significantly affected the results. No significant factors were found. These findings support the notion that ServSafe Online is an effective method of instruction. Keywords: Online education, ServSafe, foodservice training. INTRODUCTION The primary purpose of this research is to determine the learning effectiveness of ServSafe Online. The National Restaurant Association Educational Foundation (NRAEF) introduced its Internet-based ServSafe Manager Certification Training Online Course (ServSafe Online) in the fall of 2003. As technological capabilities and demand for the convenience of delivering training online continued to grow in the foodservice industry, NRAEF wished to assess the effectiveness of the course in preparing learners for its ServSafe Food Protection Manager Certification Examination (ServSafe Exam). Sanitation certification has taken on an increasingly important role in foodservice training in the United States. The U.S. Food and Drug Administration (FDA) recommends that the person in charge at an establishment service food demonstrate knowledge of foodborne disease prevention, application of Hazard Analysis Critical Control Point principles (HACCP), and the requirements of the FDA Food Code. Managers can become a certified food protection manager through demonstrating proficiency in the required information by passing a test that is part of an accredited program. Numerous states, counties, and municipalities have passed regulations that require a certified manager to be present at foodservice establishments. The ServSafe Food Safety Manager Training and Certification Program is the leading program of instruction and certification in this area. The National Restaurant Association Educational Foundation (NRAEF), which offers this training program and examination, certified over 300,000 learners in 2005. Traditionally, this course is taught using a lecture-based format in a traditional classroom setting with a registered instructor. In the fall of 2003, however, the NRAEF introduced an online version of this course. Many articles have been written about the critical need for safe food handling instruction in the U.S. and around the world (Mortimore, 2001; Pansiello & Quantick, 2001; Sun & Ockerman, 2005; Walker, Pritchard & Forsythe, 2003), but there has been only a limited amount of research done in the area of computer-based foodservice training – one of the motivations and secondary purpose for the current study. Initial research on small samples has shown that training using technology effectively increases food safety knowledge and behavior (Eckerman, Abrahamson, Ammerman, Fercho, Rohlman, & Anger, 2004); thus, we have reason to believe that computer-based training regarding ServSafe may help improve the knowledge of foodservice managers and employees. The remainder of the paper will sequentially discuss the relevant literature regarding online education, the methodology, assessment instrument, and the results of the mailto:andy.feinstein@unlv.edu mailto:Michael.dalbor@unlv.edu mailto:mcmanus4@unlv.nevada.edu Developments in Business Simulation and Experiential Learning, Volume 34, 2007 179 study. The paper concludes with recommendations for future research. LITERATURE REVIEW The review of the literature will begin by first examining the comparative studies that have focused on online and classroom education, including reference to the limitations of these studies. The literature review will then focus on the progress of online tools over time, as well as the factors that leverage the success of online educational experiences. Finally, a review of food safety studies that specifically utilize technology in the area of food safety education is presented. A significant amount of research, writing, and publications have explored the effectiveness of online education; with much of it comparing traditional classroom training techniques with those involving computer technologies. Several researchers (e.g., Feinstein, 2004; Russell, 1999), however, believe that the methodology used in such studies is flawed. The primary reason for this belief is that the statistical approaches used to analyze the data collected require all extraneous variables (i.e., variables beyond those being studied) to be held relatively constant, so as to achieve a controlled environment. In the real world of hospitality – especially foodservice – such control is virtually impossible. The failure, and inability, to control the extraneous variables may explain the conflicting results found in the literature. Nevertheless, this literature is worthy of examination. A good example of comparative work focusing on online and traditional learning is the research by Dellana, Collins and West (2000). This study investigated student performance in a traditional classroom course and an online course and found no significant difference between the two learning approaches. The success in online courses was predicted by (1) each student’s GPA and (2) each student’s attendance rate. These success factors match those found in the study’s traditional course. Attendance rates were higher in the traditional course, but this is not surprising. Having an instructor at each meeting face-to-face elicits different behavior routines from students in general, from elementary school to college and beyond. In a traditional classroom setting, students often attempt to hide behind anything that would break eye- contact with the instructor (e.g., a tall student sitting in front of him or her). Learning in an online environment is no different. Procrastination, even if one is sitting in front of an LCD monitor, is a similar, expected, and typical behavior in many learning environments. Other researchers have suggested that online courses can not provide everything students needed to learn – but only when designed without considering the possible outcomes (Sweeney & Ingram, 2001). A number of other studies have also investigated whether online classes are more effective that traditional classes. Zhang, Zhao, Zhou and Nunamaker, Jr. (2004) compared an interactive learning environment with a virtual mentor to a traditional classroom and found that the e- learning group outperformed the classroom group. A number of other studies also found that online learners outperformed traditional classroom learners (e.g., Mao & Brown, 2005; Vachris, 1999; Zhang, Zhao, Zhou & Nunamaker Jr., 2004). However, a study by Brown & Liedholm (2002) found the online approach to be inferior to the traditional teaching approach while Picccoli, Ahmad, & Ives (2001) found no significant difference between the approaches. Mixed results of this nature leads to a state of puzzlement. There also has been some work done regarding learner characteristics that influence the level of effectiveness of online education. Such research has found that the following student characteristics are crucial determinants of online performance: Flexibility with work and family (Marks, Sibley & Arbaugh, 2005; Parnell & Carraher, 2003), age (Brown, 2001; Parnell & Carraher, 2003), gender (Marks, Sibley & Arbaugh, 2005), education, and computer experience (Brown, 2001; Marks, Sibley & Arbaugh, 2005). Even more perplexing is research that initially finds no significant difference in effectiveness between the two approaches to learning but, after controlling for certain factors, obtains results that show the online learning environment to be inferior to traditional classrooms (e.g., see Anstine & Skidmore, 2005). Results of this nature may indicate that there is an interaction between the learning environment and the learning material: Certain subject areas may be more suitable for the online environment and others more suitable for the face-to-face environment. Past research clearly indicates there is a lack of consensus with respect to online learning outcomes; it seems that there are some good, some bad, and some ugly (Parks, 2004). Thus, the need for further investigation in this area of pedagogy is sufficient motivation for the current research study. Many articles have been written about the critical need for safe food handling instruction in the U.S. and around the world (Mortimore, 2001; Pansiello & Quantick, 2001; Sun & Ockerman, 2005; Walker, Pritchard & Forsythe, 2003), but there has been only a limited amount of research done in the area of computer-based foodservice training – another one of the motivations for the current study. Initial research, based on small respondent sample sizes, has shown that training using computer technology can increase the learner’s level of food safety knowledge (Eckerman, Abrahamson, Ammerman, Fercho, Rohlman, & Anger, 2004), thereby providing evidence that computer-based training with respect to ServSafe, should lead to the same result for participating foodservice managers and employees. PURPOSE OF THE PROJECT The purpose of this study is to determine the learning effectiveness of ServSafe Online. The research questions of this study are: Developments in Business Simulation and Experiential Learning, Volume 34, 2007 180 1. Is there a significant increase in learners’ food safety knowledge as a result of taking ServSafe Online? 2. Does ServSafe Online adequately prepare learners for the ServSafe Exam? 3. Do ServSafe Online learners pass the ServSafe Exam as frequently as those who take the traditional classroom form of instruction? 4. Are there any characteristics of learners that might explain any differences in exam scores? 5. Do users of ServSafe Online significantly increase ServSafe Exam scores (gain scores) in any of the knowledge domains described below? METHODOLOGY PARTICIPANTS Participants in this study were drawn from three sources: the St. Joseph County, Indiana Health Department (foodservice industry workers), the University of Minnesota Extension Service (foodservice industry workers), and the University of Nevada, Las Vegas (students in the William F. Harrah College Hotel of Hotel Administration, more than half of whom also have industry work experience). Selected demographic characteristics of the subjects are presented in Appendix 1. The subjects were not paid for their participation, as it was thought this might change the nature of the sample and reduce its generalizability to the population taking ServSafe Online and the ServSafe Exam. INSTRUMENT The study relied upon the ServSafe Exam, an exam that is accredited by the American National Standards Institute (ANSI) and the Conference for Food Protection (CFP) and has been used and regularly updated for over a decade. This examination consists of ninety, 4-option multiple-choice questions containing 80 operational and ten pilot items randomly placed within the examination (National Restaurant Association Educational Foundation, 2006). For purposes of the study, the ten pilot items were eliminated from the analysis. The exam is designed to assess the knowledge required of food protection managers to protect the public from food borne illness and is broken down into content domains. We tested two domain structured exams in this study – one containing seven (7) domains, and one containing ten (10). The domain structures of the two exams are shown in Table 1 below. DATA COLLECTION Before taking the SafeServ exam, participants first signed a consent form required by UNLV and completed a demographic profile survey. After completing these tasks, each participant was assigned a login user-ID and a password for ServSafe Online and then took the ServSafe Exam to assess the individual’s level of food safety knowledge before having the opportunity to take the actual online course. The exam was administered in a secure, proctored environment, supervised by registered ServSafe instructors/proctors, in accordance with NRAEF specifications. All participants subsequently took the online course, accessing it through the NRAEF website using the login user-ID and password provided. The time allotted to complete the course varied by organization, ranging from two weeks to sixty days. The learners were free to use any Internet-enabled computer they chose – personal/home computer, work computer, school or library (computer specifications for optimal course performance were provided). Technical support was available through NRAEF Customer Service. The average time required for the participants to complete the course was 8 hours. After completing the online course, the participants took a version of the ServSafe Exam that was different than the one taken before content instruction. However, both versions were ANSI accredited for consistency. The exam was, once again supervised by registered ServSafe instructors/proctors in a secure environment. The pre- and post-exams were administered in paper and pencil form, since the online version of the exam was not available at the Table 1 Domains Assessed Seven-Domain Structure 1. Ensure Food Protection 5. Serve and Display Foods 2. Purchase and Receive Food 6. Maintain Equipment and Supplies 3. Store Food and Supplies 7. Monitor Food Personnel 4. Prepare Foods Ten-Domain Structure 1. Foods 6. Allergens 2. Clean, Sanitize, and Maintain Equipment 7. High-Risk Populations 3. Facilities 8. Legal and Regulatory Issues 4. Personnel 9. Facility Layout and Design 5. Temperature Measurement Devices 10. Train Employees Developments in Business Simulation and Experiential Learning, Volume 34, 2007 181 onset of the study. ANALYSES We attempted to assess if any significant differences in ServSafe Exam scores exist between a pre-test and a post- test. We chose this method because we were unable to organize a control group and compare their scores to other students. Furthermore, rather than calculate a simple raw difference between pre- and post-test, we created a percentage gain score between pre- and post-test. This takes into account that some participants started out knowing more of the ServSafe Online material than others and more accurately measures the relative participant acquisition of material resulting from the ServSafe Online course. Of the 523 study participants who took the pre-test, 84 did not take the post-test. The latter group of students were removed from the data sample. Another 48 subjects did not take the same form (7-domain, 10-domain) of the test for their pre-test/post-test pairing. These subjects were also excluded. Another 48 subjects who scored a passing grade (75% or higher) on the pre-test were excluded, since they showed sufficient comprehension of the content to be certified at the study’s onset. The remaining 343 subjects were used in the computation and analysis of the results. We first compared results from the two domain structured examinations to determine if participants scored differently on these two exams. Independent-samples t-tests revealed no significant differences between mean pre-test scores on the seven-domain and ten-domain ServSafe Exam structures and between the learning gains (computed as the Table 2 Descriptive Statistics, Domain Structures by Pre-Test Score Test Type N Mean Std. Deviation Std. Error Mean Pre-Test Score 7 Domains 160 58.188 11.297 0.893 10 Domains 183 58.601 11.961 0.884 Table 3 Independent-Samples T-Test, Domain Structures by Pre-Test Score Levene's Test T-Test F Sig. T df Sig. (2-tailed) Mean Difference Std. Error Difference Equal Variances Assumed 0.803 0.371 -0.328 341 0.743 -0.414 1.262 Pre-Test Score Equal Variances Not Assumed -0.329 339 0.742 -0.414 1.257 Table 4 Domain Structures by Learning Gain Score Test Type N Mean Std. Deviation Std. Error Mean Learning Gain Score 7 Domains 160 0.507 0.423 0.033 10 Domains 183 0.417 0.437 0.032 Table 5 Independent-Samples T-Test, Domain Structures by Learning Gain Score Levene's Test T-Test F Sig. T df Sig., 2-tailed Mean Difference Std. Error Difference Equal Variances Assumed 0.005 0.943 1.935 341 0.054 0.090 0.047 Learning Gain Score Equal Variances Not Assumed 1.939 337 0.053 0.090 0.047 Developments in Business Simulation and Experiential Learning, Volume 34, 2007 182 difference between post-test and pre-test score, divided by pre-test score) on these respective structures (Tables 2 - 5). Table 6 shows the details of online participant learning. The mean pre-test score of 58.41 percent (a failing grade) for the selected group of students, indicates that participants did not have sufficient knowledge before taking the course to pass the ServSafe Exam. The average grade after participating in the online course was 81.23%, an increase of over twenty-two (22) percentage points, or relative gain of 39.07 percent over the pre-test. The mean post-test score of 81.23 percent signifies that participants did not simply pass the examination, but scored over six (6) points higher than the standard examination cutoff of 75 percent. Moreover, 81 percent of participants passed the ServSafe Exam, a value similar to that obtained in 2005 (i.e., 79%) using the traditional classroom approach. It was first hypothesized that ServSafe Online would significantly increase participant’s knowledge of food safety. The results support this hypothesis. Paired sample t- test results revealed that there was a significant difference between groups (t (342) = 35.836, p < .001; see Table 6 below), meaning that participants learned a significant amount through Servsafe Online. As can be seen in Table 7, the test scores of the Table 6 Pre-Test and Post-Test Scores 58.41 343 11.641 .629 81.23 343 8.598 .464 PreTestScore PostTestScore Pair 1 Mean N Std. Deviation Std. Error Mean Table 7 Paired-Sample T-test, Pre-Test and Post-Test Scores Paired Differences Mean Std. Deviation Std. Error Mean t df Sig. (2- tailed) Pair 1 Pre-Test and Post-Test Scores 22.825 11.796 0.637 35.836 342 0.000 Figure 1 GLM Assessing Moderating Effects of Descriptive Data on Participant Gain Scores Y = μ + Agei + Genderj + Educationk + FoodServExpl + StudentStatm+ EmpFoodServn + HoursWrkWklyo + JobPositionp + MgmntExpq + ComputerCmfrtr + FoodSftyCerts + Ethnicityt + ε Where: Y = Response for ijklmnopqrst – th individual μ = Overall Mean Agei = Fixed Effect, i = 1, 2, 3, 4, 5, 6, 7, 8, 9 (<21, 21 to 25, 26 to 29, 30 to 34, 35 to 39, 40 to 44, 45 to 49, 50 to 54, 55 or older) Genderj = Fixed Effect, j = 1, 2 (Male, Female) Educationk = Fixed Effect, k = 1, 2, 3, 4, 5, 6 (Some High School, High School Graduate, Some College, Community College or trade School Degree, Four Year College Degree, Graduate Degree) FoodServExpl = Fixed Effect, l = 1, 2, 3, 4 (0, < 1, ≥1 but < 3, > 3) StudentStatusm = Fixed Effect, m = 1, 2, 3 (No, Full Time, Part Time) EmpFoodServn = Fixed Effect, n = 1, 2 (Yes, No) HoursWrkWklyo = Fixed Effect, o = 1, 2, 3, 4 (< 20, 20 to 29, 30 to 39, ≥ 40) JobPositionp = Fixed Effect, p = 1 through 12 (Server, Cook, Chef, Manager, Supervisor, F&B Director, Regional Manager, General Manager, Owner, Corporate Executive, Trainer/Instructor, Other) MgmntExpq = Fixed Effect, q = 1, 2, 3, 4 (0, < 1, ≥1 but < 3, > 3) ComputerCmfrtr = Fixed Effect, r = 1, 2, 3, 4, 5 (Very Uncomfortable, Somewhat Uncomfortable, Neutral, Somewhat Comfortable, Very Comfortable) FoodSftyCerts = Fixed Effect, s = 1, 2 (No, Yes) Ethnicityt = Fixed Effect, t = 1, 2, 3, 4, 5, 6 (Asian or Pacific Islander, Black/African-American, Hispanic, Native American, White/Caucasian, Other) ε = Error Term = All two way and higher interactions Developments in Business Simulation and Experiential Learning, Volume 34, 2007 183 participants increased very substantially—over 22 points. Additionally, the average pre-test score was a failing grade while the average post-test score was a passing grade. In an effort to further explain the gains in the group’s mean post-test scores over its mean pre-test scores, a second phase of analysis examined whether demographic characteristics identified homogeneous traits in some of the participants within their respective group; that is, do participants with certain demographic characteristics respond to ServSafe Online differently, as evidenced by the level of learning? The different demographic traits (see Appendix 1), or moderating variables, were statistically Table 8 One-Way ANOVA, Ethnicity and Learning Gain Score .781 5 .156 9.908 .000 4.776 303 .016 5.557 308 Between Groups Within Groups Total Sum of Squares df Mean Square F Sig. Table 9 SAMPLE SIZES FOR GROUPS DELINEATED BY ETHNICITY 103 .5490 .14492 .01428 12 .3936 .13886 .04008 15 .5040 .11003 .02841 3 .3670 .05019 .02897 164 .4490 .11351 .00886 12 .4777 .11555 .03336 309 .4832 .13432 .00764 Asian or Pacific Islander Black/African-American Hispanics Native American White/Caucasian Other Total N Mean Std. Deviation Std. Error Figure 2 Plot of Gain Score Means by Ethnicity Asian African American Hispanic Native American Caucasian Other 35% 40% 45% 50% 55% Ethnicity M ea n Pe rc en ta ge G ai n Developments in Business Simulation and Experiential Learning, Volume 34, 2007 184 compared to gain scores (post-test score minus pre-test score divided by pre-test score) transformed using an arc sine transformation. The calculation for this transformation is as follows: p1sin*2 −=Φ with the angle p expressed in radians The gain score, as the dependent variable, must be transformed by converting the scores from proportional to continuous data in order to obtain data in a form that fulfills one of the basic assumptions of General Linear Models (GLM). To determine a transformation technique that would yield a high consistency of error variance, data were transformed using two methods: arc sine and logit. After careful review, it was determined that the arc sine transformation yielded the best results. The analysis also called for the calculation of Pearson’s Product-Moment Correlations using the moderating variables in order to identify and minimize any multicolinearity that may be present. Based upon these steps, the model, as described in Figure 1, was constructed. The results of the GLM indicate that ethnicity (F (5) = 3.557, p = 0.004) is the only significant variable moderating gain scores at the α = .05 significance level. Therefore, it was our initial concern that ethnicity may moderate the effectiveness of ServSafe Online. Because ethnicity was identified as a significant moderator of ServSafe Online’s learning effect, we investigated further, using a one-way ANOVA test (Table 8) and a plot of mean scores for ethnic groups (Table 9 and Figure 2) to graphically depict what various participants gained using ServSafe Online. The one-way ANOVA test confirms previously suggested relationships between ethnicity and learning (see Enoch and Soker, 2006); but despite the previous statistical analyses, it does not conclusively mean that ServSafe Online inherently differentiates between participants of varying ethnicities. It is highly likely that there are two factors at work: (1) that the sample here includes some ethnic groups of such low numbers that it skews statistical results and (2) that there may be underlying or latent variables at work – that is, it may not be ethnicity, but another hidden variable, that truly caused the effect seen. In order to answer questions about the relationships between ethnicities and learning, a study with a greater number of each ethnicity would need to be conducted. An alternative statistical approach to this issue would be to analyze the data using nonparametric statistics; thus, a Chi-square test (Table 10) was conducted on the ethnicity variable alone. Results again suggested that varying ethnic groups experienced significantly different learning gains from ServSafe Online from both statistical analyses. Table 10 Chi Square Test, Ethnicity and Learning Gain Score 12.026 5 .034 Chi-Square df Asymp. Sig. PostTest Next, we investigated the possibility that significant differences may be due to a skewed gain score (the difference between post-test score and pre-test score, divided by pre-test score) measurement. This could be caused by (1) a significantly higher pre-test score by participants in some sub-groups, (2) a significantly lower post-test score by participants in some sub-groups, or (3) a combination of the two. A quick examination of pre-test scores shows that skew is a likely contributing factor. In Table 11, it is clear that the Asian/Pacific Islander sub-group had a significantly lower pre-test score mean, at approximately 52%; while the Native American sub-group’s pre-test score averaged about 71%. This would limit the amount of knowledge that could be gained of the Native American sub-group compared to the amount that the Asian/Pacific Islander sub-group would have. Due to the high probability of skew in the gain score measurement and the considerably low numbers of participants in some ethnic sub-groups, it cannot be concluded that some ethnic groups learned significantly more or less than others using the online course. The only conclusive finding we can present from these investigations with statistical certainty is that the online course effectively increased ServSafe Exam scores, regardless of one’s ethnic background, to the passing range. Table 11 Mean Pretest Scores by Ethnicity Ethnicity Mean N Std. Deviation Asian or Pacific Islander 51.87 104 11.929 African American 59.17 12 9.331 Hispanic 58.81 16 9.174 Native American 71.33 3 2.309 White/Caucasian 63.14 166 8.601 Other 60.08 12 12.325 Total 58.98 313 11.235 CONCLUSIONS AND IMPLICATIONS FOR FURTHER RESEARCH The purpose of this study was to assess if ServSafe Online is an effective learning tool. The answer, as was discussed in this report, is unequivocally yes, according to the results from our sample. The mean scores for the Developments in Business Simulation and Experiential Learning, Volume 34, 2007 185 ServSafe Exam improved 22 points, from a failing grade to a passing one. Additionally, we examined the potential effects of a variety of demographic factors and the results were not shown to be affected by any demographic demarcation found in other research. The results of the study indicate that online learning was shown to significantly increase exam scores when comparing the post-test scores to the pre-test scores across all participant demographic groups examined. The relationship between ethnicity and online learning should be investigated in future studies, ensuring that appropriate ethnic group sample sizes are used. A similar analysis should also compare the learning results of those who have had previous foodservice or industry experience to those who have had not such experience. It may be that there are different training needs based on the level of experience. Further research may target those demographic groups of limited representation in this sample. The most notable of these groups are (a) African-American participants (by statistical identification), and (b) those with previous foodservice or industry experience (of non-statistical but practical importance). Determining the relative worth or value of the different domain structures (i.e., 7-domain versus 10-domain) remains open to future investigation. To solidify the analysis and subsequent findings presented here, a comprehensive domain analysis is recommended, if possible. This action would further improve the reliability and validity of the findings presented here. REFERENCES Anstine, J. & Skidmore, M. (2005). A small sample study of traditional and online courses with selection adjustment. Journal of Economic Education 36, 107- 127. Brown, K. (2001). Using computers to deliver training: Which employees learn and why? Personnel Psychology 54, 271-296. Brown, A. & Liedholm, C. (2002). Can web courses replace the classroom in principles of Economics? American Economic Review 92, 444-449. Dellana, S., Collins, W. & West, D. (2000). On-line education in a management science course— effectiveness and performance factors. Journal of Education for Business 76, 43-47. Eckerman, D., Abrahamson, K. Ammerman, T., Fercho, H., Rohlman, D. & Anger, K. (2004). Computer-based training for food services workers at a hospital. Journal of Safety Research 35, 317-327. Enoch, Y. & Soker, Z. (2006). Age, gender, ethnicity and the digital divide: university students’ use of web- based instruction. Open Learning 21, 99-110. Feinstein, A. (2004). A model for evaluating online instruction. Developments in Business Simulation and Experiential Learning 31, 32-39. Mao, J. & Brown, B. (2005). The effectiveness of online task support vs. instructor-led training. Journal of Organizational and End User Computing 17, 27-46. Marks, R., Sibley, S. & Arbaugh, J. (2005). A structural equation model of predictors for effective online learning. Journal of Management Education 29, 531- 563. Mortimore, S. (2001). How to make HACCP really work in practice. Food Control 12, 209-215. National Restaurant Association Educational Foundation. (2006). ServSafe® Examination Administration Handbook. Retrieved December 15, 2006 from http://www.nraef.org/pdf_files/ExamHdbk_Links.pdf. Pansiello, P. & Quantick, P. (2001). Technical barriers to hazard analysis and critical control point (HACCP). Food Control 12, 165-173. Parks, R. (1999). Macro principles, PowerPoint and the Internet: four years of the good, the bad and the ugly. Journal of Economic Education 30, 200-209. Parnell, J. & Carraher, S. (2003). The management education by Internet readiness (MEBIR) scale: developing a scale to assess personal readiness for internet-mediated education. Journal of Management Education 27, 431-446. Piccoli, G., Ahmad, R. & Ives, B. (2001). Web-based virtual learning environments: A research framework and a preliminary assessment of effectiveness in basic IT skills training. MIS Quarterly 25(4), 401-426. Russell, T.L. (1999). The No Significant Difference Phenomenon. Raleigh, NC: Office of Instructional Communications, North Carolina State University. Sun, Y. & Ockerman, H. (2005). A review of the needs and current applications of hazard analysis and critical control point (HACCP) system in food service areas. Food Control 16, 325-332. Sweeney, J. & Ingram, D. (2001). A comparison of traditional and web-based tutorials in marketing education: An exploratory study. Journal of Marketing Education, 23(1), 55-62. Vachris, M. (1999). Teaching principles of economics without “chalk and talk”: The experience at CNU on- line. Journal of Economic Education 30, 292-302. Walker, E., Pritchard, C. & Forsythe, S. (2003). Hazard analysis critical control point and prerequisite programme implementation in small and medium size food business. Food Control 14, 169-174. Zhang, D., Zhao, J., Zhou, L. & Nunamaker, Jr., J. (2004). Can e-learning replace classroom learning? Communications of the ACM 47, 75-79. http://www.nraef.org/pdf_files/ExamHdbk_Links.pdf Developments in Business Simulation and Experiential Learning, Volume 34, 2007 186 APPENDIX 1 DESCRIPTIVE INFORMATION OF PARTICIPANTS Table A: Descriptives of Participants by Age 135 39.4 39.6 39.6 134 39.1 39.3 78.9 25 7.3 7.3 86.2 18 5.2 5.3 91.5 6 1.7 1.8 93.3 8 2.3 2.3 95.6 5 1.5 1.5 97.1 7 2.0 2.1 99.1 3 .9 .9 100.0 341 99.4 100.0 2 .6 343 100.0 less than 21 years old 21 to 25 26 to 29 30 to 34 35 to 39 40 to 44 45 to 49 50 to 54 55 or older Total Valid SystemMissing Total Frequency Percent Valid Percent Cumulative Percent Table B: Descriptives of Participants by Gender 139 40.5 40.6 40.6 203 59.2 59.4 100.0 342 99.7 100.0 1 .3 343 100.0 Male Female Total Valid SystemMissing Total Frequency Percent Valid Percent Cumulative Percent Table C: Descriptives of Participants by Education 2 .6 .6 .6 64 18.7 18.7 19.3 187 54.5 54.7 74.0 47 13.7 13.7 87.7 35 10.2 10.2 98.0 7 2.0 2.0 100.0 342 99.7 100.0 1 .3 343 100.0 Some High School High School Graduate Some College Community College or Trade School Degree Four Year collegedegree Graduate degree Total Valid SystemMissing Total Frequency Percent Valid Percent Cumulative Percent Developments in Business Simulation and Experiential Learning, Volume 34, 2007 187 Table D: Descriptives of Participants by Previous Foodservice Experience p 144 42.0 42.2 42.2 44 12.8 12.9 55.1 67 19.5 19.6 74.8 86 25.1 25.2 100.0 341 99.4 100.0 2 .6 343 100.0 None Less than 1 year One year but less than 3 Three or more years Total Valid SystemMissing Total Frequency Percent Valid Percent Cumulative Percent Table E: Descriptives of Participants by Student Status 41 12.0 12.1 12.1 280 81.6 82.4 94.4 19 5.5 5.6 100.0 340 99.1 100.0 3 .9 343 100.0 No Yes - Full time student Yes - Part time student Total Valid SystemMissing Total Frequency Percent Valid Percent Cumulative Percent Table F: Descriptives of Participants by Current Foodservice Employment 90 26.2 26.5 26.5 250 72.9 73.5 100.0 340 99.1 100.0 3 .9 343 100.0 Yes No Total Valid SystemMissing Total Frequency Percent Valid Percent Cumulative Percent Table G: Descriptives of Participants by Foodservice Hours Worked Per Week 18 5.2 19.4 19.4 19 5.5 20.4 39.8 26 7.6 28.0 67.7 30 8.7 32.3 100.0 93 27.1 100.0 250 72.9 343 100.0 less than 20 20 to 29 hours 30 to 39 hours 40 hours or more Total Valid SystemMissing Total Frequency Percent Valid Percent Cumulative Percent Developments in Business Simulation and Experiential Learning, Volume 34, 2007 188 Table H: Descriptives of Participants by Current Foodservice Position 19 5.5 20.4 20.4 6 1.7 6.5 26.9 1 .3 1.1 28.0 22 6.4 23.7 51.6 2 .6 2.2 53.8 7 2.0 7.5 61.3 6 1.7 6.5 67.7 2 .6 2.2 69.9 1 .3 1.1 71.0 27 7.9 29.0 100.0 93 27.1 100.0 250 72.9 343 100.0 Server Cook Chef Manager Supervisor General Manager Owner Corporate Executive Trainer/Instructor Other Total Valid SystemMissing Total Frequency Percent Valid Percent Cumulative Percent Table I: Descriptives of Participants by Previous Foodservice Management Experience 224 65.3 69.6 69.6 34 9.9 10.6 80.1 36 10.5 11.2 91.3 28 8.2 8.7 100.0 322 93.9 100.0 21 6.1 343 100.0 None Less than one year One year or more but less than 3 years Three or more years Total Valid SystemMissing Total Frequency Percent Valid Percent Cumulative Percent Table J: Descriptives of Participants by Ease with Using a Computer 81 23.6 23.8 23.8 28 8.2 8.2 32.1 37 10.8 10.9 42.9 98 28.6 28.8 71.8 96 28.0 28.2 100.0 340 99.1 100.0 3 .9 343 100.0 Very Uncomfortable Somewhat uncomfortable Neutral Somewhat comfortable Very Comfortable Total Valid SystemMissing Total Frequency Percent Valid Percent Cumulative Percent Developments in Business Simulation and Experiential Learning, Volume 34, 2007 189 Table K: Descriptives of Participants by Previous Food Safety Certification Testing y 302 88.0 88.6 88.6 39 11.4 11.4 100.0 341 99.4 100.0 2 .6 343 100.0 No Yes Total Valid SystemMissing Total Frequency Percent Valid Percent Cumulative Percent Table of Contents Volume 34, 2007 Experiential Teaching May Lead To Experiential Learning The Role Of Learning Versus Performance Orientations When Reacting To Negative Outcomes In Simulation Games: Further Insights An Analysis Of The Interaction Of Firm Demand And Industry Demand In Business Simulations Consistency Of Participant Simulation Performance Across Simulation Games Of Growing Complexity Assessing And Incentivizing Learner Contribution And Performance Excellence: Creating Win-Win Evaluation Options The Technological Impact Analysis: A Research-Based Exercise To Heighten Learners' Technological Sensitivity Beginnings: How We Start the Semester and Individual Classes : A Roundtable Discussion Simulation Performance and Its Effectiveness As A PBL Problem: A Follow-Up Study The Use of Multimedia Learning Tools to Facilitate Online Learning of Business Statistics Forecasting Accuracy And Learning: A Key To Measuring Business Game Performance Panel Discussion: Alternative Ways of Using the Internet for Business Simulations to Input Decisions, Process, and Present Financial and Economic Data Output Absel Research -- One Additional Perspective On Where We Are And Where We Have Come From Assessing participant learning in a business simulation From Case Presentation To Case Facilitation: How Assessment Changed The Capstone Course The Use Of Computer-Assisted, Interactive Role-Play Simulation In Hong Kong Corporate Positioning: A Business Game Perspective Demonstration Of A Computer-Assisted Global Business Simulation Applying .NET Remoting To A Business Simulation Assessing Emotional Intelligence: The EQ Matrix Exercise Outcomes And Observations Of An Extended Accounting Board Game The Use Of Learning Styles Questionnaire In Hong Kong Forming Teams For Classroom Projects Online Budgeting And Marketing Control With The Proforma Analysis Package The Effect Of Experiential Learning Experiences On Management Skills Acquisition Students' Perceptions on the Individual Managerial Performance Orderness: A New Definition Of Alignment An Exploratory Study Of The Efficacy Of Servsafe® Online Team Behavior And Team Success Results From A Board Game Simulation Telecommuting Internships - Do They Work? The WEE GAME: A Pre-Game Effects On Learning When Students Have Information About Games And Their Outcomes When Playing Them Assessment And Simulations: Measuring The Academic Learning Compacts Within Dangerous Business: An Interactive Ethics Case Activity The Paradise Islands Assessing And Applying fiAppropriatefl Conflict Management Styles Distance Learning In Communication: Blended Or On-Line? Developing An On-Line Advanced Communication Course World Café: Simulating Seminar Dialogues in a Large Class The Programming Game: An Exploratory Collaboration Between Business Simulation And Instructional Design The Canary Principle: An Alternative Model for Providing Real-Time Coaching in an On-Line Discussion Environment Assessing Character Development Using A Direct-Approach Business Ethics Exercise Persistence Of Decisions By Simulation Game Participants The Invalidity of Profit = f(Product Quality) PIMS Validation of Marketing Games A Study of The Applicability of The Perceptions of Organizational Politics Scale (POPS) For Use in The University Classroom The ROI Of Connected Projects: A Payroll Example Initiation of Research on Gaming Simulation in Japan Inappropriate Use of Citations and Corrupting the Body of Knowledge: Accepting Urban Legends as Truth An Exploration Of The Perceived Value Of Highly Socio-Inductive Learning How "Whole" Is Whole Person Learning? An Examination Of Spirituality In Experiential Learning Modeling Outsourcing and Strategy Alignment into a Business Game Successful Integration Of Webct Into A Small Business School The Thinking Steps Model In Game Theory: A Qualitative Approach In Following The Rules The Application Of Means-End Theory To Understanding The Value Of Simulation-Based Learning Armchair Travel: An active learning approach to increasing global awareness and participant self-efficacy Goldratt's Thinking Process: Is there a place for it in the Total Enterprise Simulation Assessing And Developing Student Skills Using a Group Exercise Computer Business Simulation Design: Novelty & Complexity Issues