Palmer, S. 70 Understanding the Context of Distance Students: Differences in On- and Off-campus Engagement with an Online Learning Environment Stuart Palmer, Deakin University Abstract While a growing number of higher education institutions are providing online modes of study for both on- and off-campus students, do the differences in demography, technology experiences and reasons for study between on- and off-campus students mean that they interact differently with online learning environments (OLEs)? Australia’s Deakin University is a major provider of distance and online education that provides a case study of a higher education institution with a mature and large-scale OLE implementation providing support to both on- and off-campus students. Deakin Studies Online (DSO) is Deakin University’s OLE. Based on a representative sample of 1322 responses to the 2011 DSO evaluation survey, this paper presents a large-scale, up-to-date and fine-grained investigation of the impact of mode of study on the student experience of using an OLE. It was found that the primary place of access to DSO for both groups was home; mobile access to DSO seems likely to be of growing importance to both groups; and there was no statistically significant difference in the mean satisfaction ratings between on- and off-campus students for virtually all DSO functions. Off-campus students gave significantly higher mean ratings of importance (though not satisfaction) to a range of DSO functions that could be viewed as ‘value adders’, enhancing their overall learning experience. For more than half of the DSO functions surveyed, on-campus students reported statistically significant higher mean frequency of access than off-campus students. The finding that elements of the institutional OLE are not universally perceived and used the same way by all student groups challenges the value of standard, ‘one-size-fits-all’ institutional policies and templates relating to the use of OLEs. Keywords: online learning; distance education; online learning environment Introduction Distance education, in many places and forms, and for many purposes, continues to thrive. This is in part due to a complex relationship with a range of online educational technologies that offer new affordances to, and challenges for, educators (Calvert, 2005; Larreamendy-Joerns & Leinhardt, 2006; Saba, 2005). Online learning environments (OLEs) are perhaps currently the most widely used and most expensive educational technology tool (Salinas, 2008; West, Waddoups, & Graham, 2007). Like many other learning technology trends before them, they have been adopted by higher education institutions almost automatically and often uncritically (Reynolds, Treharne, & Tripp, 2003). Much of the research into online learning seems to assume Journal of Open, Flexible, and Distance Learning, 16(1) 71 no influence from the demographic characteristics of system users (Woods, Baker, & Hopper, 2004). While a growing number of higher education institutions are providing online modes of study for both on- and off-campus students, there are very real differences in demography, technology experiences, and reasons for study between on- and off-campus students. Research into engagement with OLEs indicates differences in the way that on- and off-campus students interact with them (Xie, Lin, & Zhang, 2001). The need to explore these differences at a high level of granularity of OLE functionality has been highlighted, lest the details of any differences are lost/‘washed out’ by too high a level of aggregation (Martz & Shepherd, 2007). In Australia, Deakin University is a major provider of distance and online education. In recent times the university has implemented institution-wide online teaching and learning systems to provide opportunities to bring together all students in the one learning community. Deakin University is a case study of a higher education institution with a mature and large-scale OLE implementation providing support to both on- and off-campus students who are, in many cases, enrolled in the same units of study and studying the same learning materials at the same time. Previous smaller-scale investigations at Deakin University identified differences in the perceptions and use of elements of the OLE between on- and off-campus students (Palmer, 2001). However, a significant period of time has passed since those previous investigations. In addition, there is very limited reporting of comparative investigations between on- and off- campus student use of the same OLE (Xie et al., 2001). This paper presents a large-scale, up-to- date and fine-grained investigation of the impact of mode of study on the student experience of using an OLE, highlights differences relating to mode of study, and identifies implications of these differences for those with policy and operational responsibilities for institutional OLEs. Methodology Deakin Studies Online (DSO) is the internal brand for Deakin University’s OLE. It includes a commercial learning management system (LMS) as the core infrastructure, as well as a range of supporting technologies, including synchronous communications, lecture recording and streaming, and plagiarism detection. During 2011, all coursework students at Deakin University, except immediately commencing students, were invited to complete the DSO evaluation survey. This population group was selected because only coursework students use DSO regularly in their studies (research students do not generally use DSO). Because the survey was conducted early in the year, immediately commencing students would have very limited experience using DSO— hence their exclusion from the survey. A total of 22,760 students were invited to complete the survey, including 6509 students enrolled in off-campus mode and 16,251 students enrolled in on- campus mode. The DSO evaluation survey sought responses from students relating to:  demographic and background information  place and mode of access to DSO  perception of importance of, satisfaction with, and frequency of use of, a range of DSO functions  a number of overall DSO satisfaction measures  open-ended written comments about DSO. The 2011 DSO evaluation survey is a refinement of previous institutional DSO evaluation instruments used at Deakin University (Palmer & Holt, 2010), and derives a measure of face validity from its repeated use and review since 2004, including successive reviews by the Deakin University Human Research Ethics Committee (DUHREC). The survey does not attempt to measure any particular theorised constructs in the respondent sample, hence its reliability relates to the accuracy with which the responses of the sample match those of the overall target population. Details about the demographic match between the respondent sample and the Palmer, S. 72 population are provided in the following discussion, and measures of rating data variation and/or confidence intervals are provided where appropriate. The current version of the DSO evaluation survey used here is included as the appendix to this paper. As required by DUHREC procedures, the survey was anonymous and voluntary. In all of the following quantitative analyses a statistical significance level of p < 0.01 has been adopted. This significance level indicates that the observed result is likely to occur by chance only once for every hundred similar respondent samples, which strongly suggests that any observed difference in mean ratings is a real difference. Results and discussion Response rate and demographic information As noted above, the reliability of the survey data depends on the accuracy with which the responses of the sample match those of the overall target population. In turn, this depends on the survey response rate, and the demographic match between the overall population and the respondent sample. An online system was used to administer the survey. It saved all progressive responses entered, resulting in differential response rates for different sections of the survey. The effective response rate for those completing the entire survey was 5.81%, although higher response rates were obtained for prior sections of the survey. A range of demographic information was available for the overall Deakin University student population as well as being collected as part of the survey, including gender, enrolled faculty, enrolled campus and duration of current enrolment. This data permitted a comparison between the respondent sample and the overall student population on these demographic dimensions, as presented in Table 1. Table 1 Response and demographic information for 2011 DSO evaluation survey Population Sample Number of respondents 22,760 1322 Gender Female 59.5% 67.8% Male 40.5% 32.2% Faculty Arts and Education 30.4% 30.1% Business and Law 36.9% 28.6% Health 19.8% 22.9% Science and Technology 12.9% 16.9% Other - 1.5% Campus Geelong – Waurn Ponds 12.1% 14.1% Geelong – Waterfront 5.8% 7.5% Melbourne – Burwood 50.3% 43.5% Warrnambool 3.2% 3.5% Off-campus 28.6% 31.4% Mean enrolment duration 2.24 years 2.42 years Journal of Open, Flexible, and Distance Learning, 16(1) 73 The survey was administered in the first teaching period of the year, and it was not possible to include newly commencing students in the population sample. However, an attempt was made to quantify the possible impact of this slightly different population group on student perceptions of online aspects of their study. Deakin University conducts an evaluation of teaching for every offer of the majority of units of study. This evaluation asks students to indicate their level of agreement (on a scale of 1–5) with a range of statements, including item 9: “The online teaching and resources in this unit enhanced my learning experience”. Commencing students are most likely to be found enrolled in first-level units of study offered in the first teaching period of the year. Using data from the Deakin University student evaluation of teaching database for an entire annual period (mid-2009 to mid-2010), the mean rating for item 9 for first-level units offered in the first teaching period was compared with the mean rating from all other units of study. The mean ratings were identical to the fourth significant figure (first level = 3.7657; other = 3.7656) and, following confirmation of homogeneity of variance, an analysis of variance (ANOVA) test indicated no significant difference between the mean ratings from both classes of units (F1427 = 3x10 -6 ; p > 0.998). Of particular interest for this investigation are the proportions of off-campus students in the population and respondent groups. By combining numbers for all campus locations into a total ‘on-campus’ figure and comparing that with the off-campus figure for both the population and sample, Fisher’s exact 2 x 2 contingency test of mode of study by group was possible, and there was no significant difference in the proportions of off-campus students between the population and the sample (p = 0.030). Although the overall response rate obtained was comparatively low, it was not unexpected for an online voluntary survey (Cook, Heath, & Thompson, 2000). The generally good match between the sample and population demographic characteristics, including the proportions of off-campus students, and the confirmation that commencing students do not hold significantly different views from other students about the value of online aspects of their study, provides some confidence in drawing more general inferences about the wider Deakin University student population from the respondent data. Two significant differences were observed in the responses to the demographic information between on- and off-campus students. The first was mean respondent age, which, for on-campus students was 24.78 years, and for off-campus students was 32.26 years. The size and variance of both groups were different, so Welch’s robust ANOVA test was performed. The observed mean ages were significantly different (F713.4 = 473.5; p < 7x10 -81 ). The second difference was mean duration of current enrolment, which, for on-campus students was 2.32 years, and for off-campus students was 2.66 years. The size and variance of both groups were different, so Welch’s robust ANOVA test was performed. The observed mean enrolment durations were significantly different (F635.4 = 14.1; p < 0.0002). These results were as expected, and lend further support to the representativeness of the respondent sample. Experience at Deakin University shows that, compared with on-campus students, off-campus students are more likely to be of a mature age and working full time while studying at less than a full load; hence they are, on average, older and take longer to complete their studies—similar cohort differences are also reported elsewhere (Young & Gibbings, 2007). No significant difference was observed between on- and off-campus students in the proportion of respondents indicating that they used DSO in their studies—99.52% of all respondents. Access to DSO Respondents were asked to choose one of three options for their principal place of access (on- or off-campus) for DSO. Based on Pearson’s chi-square test, the distribution of responses was significantly different between on- and off-campus students (Χ 2 2 = 115.2; p < 1x10 -25 ), and these are presented in Figure 1(a). Respondents were asked to indicate any of three options for where else they access DSO from. Based on Fisher’s exact 2 x 2 contingency test, the proportions of Palmer, S. 74 respondents indicating secondary places of access for DSO as “on-campus” (p < 4x10 -14 ) and “other” (p < 3x10 -14 ) were significantly different between on- and off-campus students, and are presented in Figure 1(b). 17.0% 0.2% 82.3% 93.6% 0.7% 6.2% 0% 20% 40% 60% 80% 100% On-campus students Off-campus students Other Home On-campus 75.2% 8.2% 22.8% 17.1%20.5% 68.6% 0% 20% 40% 60% 80% On-campus students Off-campus students On-campus Home Other Figure 1 (a) Primary place of access to DSO (b) Other places of access to DSO As might be expected, nearly all off-campus students indicated that their primary place of access for DSO was their home. What is perhaps surprising was that more than 80% of on-campus students also indicated their home as their primary place of access. Consideration of computing access from outside the university is no longer just the preserve of those supporting off-campus students. In terms of interacting with DSO, the objective physical student experience is likely to be largely the same regardless of the specific mode of enrolment. For secondary sources of access for DSO, off-campus students were significantly more likely to indicate “other”. While the details of the other sources are not known, experience suggests it is likely to be predominantly people’s place of work. Respondents were asked to choose one of five options for their principal mode of access (type of computing device used) for DSO. There were no significant differences in the distribution of responses between on- and off-campus students. The overall combined distribution is presented in Figure 2(a). Respondents were asked to indicate any of five options for how else they access DSO. Based on Fisher’s exact 2 x 2 contingency test, the proportions of respondents indicating secondary modes of access for DSO as “desktop computer” (p < 5x10 -7 ) and “smartphone” (p < 4x10 -7 ) were significantly different between on- and off-campus students, as presented in Figure 2(b). Journal of Open, Flexible, and Distance Learning, 16(1) 75 29.5% 69.8% 0.6% 0.1% Desktop Laptop Tablet Smartphone Other † 58.5% 44.0% 25.4% 30.5% 5.9% 5.9% 29.1% 16.6% 1.8% 1.1% 0% 20% 40% 60% On-campus students Off-campus students Desktop Laptop Tablet Smartph. Other † No responses recorded for Other Figure 2 (a) Primary mode of access to DSO (b) Other modes of access to DSO DSO is still largely primarily accessed via ‘full-function’ computers, although two-thirds of respondents indicated that a portable/laptop computer was their preference. Mobile devices barely rated as the primary method of access to DSO. Interestingly, when it came to secondary modes of access to DSO, a significant minority of all respondents indicated using a mobile device, with a smartphone being the most popular (and more popular with on-campus than off- campus students). Mobile access to DSO seems likely to be of growing importance to all students. Importance and satisfaction ratings For each of the 15 core LMS functions and 5 DSO support functions (together representing the majority of Deakin University’s current OLE functionality) listed in Figure 3, respondents were asked to indicate how important it was to the success of their studies and how satisfied they were with it. Importance was rated on a scale of 1–5: 1 = not important; 3 = neutral; and 5 = very important, and satisfaction was rated on a scale of 1–5: 1 = not satisfied; 3 = neutral; and 5 = very satisfied. A “not applicable” response option was also available. Presuming an ordinal scale of values for respondent ratings, an indication of the significance of the differences between the mean ratings for an item between on- and off-campus is obtained from an ANOVA test using mean item rating as the dependent variable and mode of enrolment as the grouping variable. A requirement for the ANOVA test is that the variation of the mean rating be similar in both enrolment modes. Levene’s test of homogeneity of variance failed for many DSO functions, and in these circumstances a robust ANOVA test using the Welch test statistic was performed instead. Based on a significance level of p < 0.01, Figure 3 summarises the mean importance and satisfaction ratings obtained. Where there was no significant difference (N.S.D.) in the mean importance and satisfaction ratings from on- and off-campus students, the overall mean ratings are plotted as a point pair (functions 2, 5, 6, 7, 14, 15, 17, 18 and 20). Where only the mean importance rating was significantly different, a vertical line is plotted, with the horizontal location given by the overall mean satisfaction rating, and the two endpoints of the line identifying the mean ratings separately for on- and off-campus students (functions 1, 3, 4, 8, 9, 11, 12, 13, 16 and 19). For one function (10) both the mean ratings of importance and satisfaction were significantly different between on- and off-campus students. In this case, a diagonal line is plotted, with the endpoints representing the mean importance and satisfaction ratings separately for on- and off-campus students. Palmer, S. 76 2 10 7 1 3 4 5 6 8 9 11 12 13 14 15 16 17 18 19 20 2.5 3.0 3.5 4.0 4.5 5.0 3.00 3.25 3.50 3.75 4.00 4.25 Im p o rt a n ce Satisfaction ♦ No Sig. Difference • On-campus students Off-campus students 1. Accessing unit guide and other unit information 11. Submitting assignments 2. Accessing unit lecture, tutorial or lab notes etc 12. Receiving feedback on assignments 3. Interacting with unit learning resources 13. Working collaboratively in a group 4. Using the unit calendar 14. Reviewing unit progress 5. Reading unit announcements 15. Using the e-portfolio 6. Contacting teachers via internal unit messaging 16. eLive synchronous communication tool 7. Contacting students via internal unit messaging 17. iLecture class lecture recording 8. Reading contributions to online discussions 18. Other iLecture recoding (eg podcasts etc) 9. Contributing to online discussions 19. Deakin’s social software 10. Completing online quizzes/tests 20. Turnitin plagiarism/originality assessment Figure 3 Mean importance and satisfaction ratings for DSO functions For all DSO functions except one, there was no significant difference in mean satisfaction ratings between on- and off-campus students. This suggests that regardless of mode of study most students are engaging with DSO in a similar way, and finding similar learning value in, the functions provided by DSO. As noted earlier, regardless of mode of enrolment most students indicated that their primary place of access to DSO was from home, so this may contribute to the general commonality of experience. The one exception was that off-campus students recorded a significantly lower mean satisfaction rating for the function “Completing online quizzes/tests”. In a study investigating student perceptions of online testing, two aspects were identified as having statistically significant differences between on- and off-campus students (Martz & Shepherd, 2007). Off-campus students were more likely to agree with the statement “I prefer taking tests with my other classmates in a traditional classroom”, and were less likely to agree with the statement “Overall, I was satisfied with my performance on the test”, even though there were no significant differences in the actual test results between on- and off-campus students. Martz & Journal of Open, Flexible, and Distance Learning, 16(1) 77 Shepherd (2007) speculate that off-campus students at an off-campus location have fewer cues and references points than their on-campus peers with which to gauge their relative online test performance. They may therefore feel less certain about their performance, even though it might be good in absolute terms. At Deakin University, on-campus students have access to laboratories with standardised computer hardware and software that is optimised to work with DSO, and which provide a high level of surety that an online quiz will function correctly. In contrast, off- campus students maintain their own computer systems, and there is a greater likelihood of compatibility problems that might cause online tests to function less reliably. This may be one of the contributors to the significantly lower satisfaction rating with online quizzes reported by off- campus students, especially if the test is formally assessed and contributes to their final unit mark. It has been shown that online quizzes can be academically beneficial for off- campus/distance students (Anderson, 2009), so more investigation into the reason(s) behind the difference between on- and off-campus student ratings for this specific DSO function is called for. Of the 10 DSO functions that recorded significantly different mean importance ratings (but not satisfaction ratings) between on- and off-campus students, off-campus students recorded lower mean ratings for the three functions “Using the unit calendar”, “Working collaboratively in a group” and “Deakin’s social software”. The unit calendar and Deakin’s social software tools were not widely used DSO functions, and this may have led to their low importance rating by off-campus students. Note that along with the e-portfolio function (15), which was also not widely used, these three functions have very low absolute mean ratings of importance and satisfaction for both on- and off-campus students. Of the DSO functions that are in common use, online group work received the lowest satisfaction rating. Potential practical problems with online group work are noted (Liu & Tsai, 2008), and it is perhaps not surprising that students who have elected to study in off-campus mode might view and rate group work as being less important than do on-campus students. For the other seven DSO functions recording a significant difference in mean importance rating between on- and off-campus students, off-campus students recorded higher mean ratings for the functions “Accessing unit guide and other unit information”, “Interacting with unit learning resources”, “Reading contributions to online discussions”, “Contributing to online discussions”, “Submitting assignments”, “Receiving feedback on assignment”, and “Live synchronous communication tool”. This group of DSO functions incorporates functions that could be viewed as ‘value adders’ by off-campus students, enhancing their overall learning experience beyond that which they would traditionally experience with printed study materials alone. Interacting with online discussions is identified, as is submission/return of assignments—traditionally a bugbear for off-campus students—where postal delivery times (internal and external to the university) might add more than a week to the turnaround time in both directions for hard copy off-campus assignments. Off-campus students also place a premium on the function of synchronous communication. These results are not inconsistent with the findings from an investigation of the factors motivating distance education student use of the internet—the most important motivators were content seeking and social contact (Stafford, 2005). This group of LMS functions includes items differentially rated by off-campus students as having relatively high importance but relatively low levels of satisfaction, including “Receiving feedback on assignments” and “eLive synchronous communication tool”. These findings suggest a potential mismatch of expectations and actual experience for off-campus students, as well as suggesting areas for action/attention that could yield improvements in the university experience for off- campus students. Palmer, S. 78 Frequency of use For each of the 15 core LMS functions and five DSO support functions listed in Figure 3, respondents were also asked to indicate, on average, how frequently they used/accessed it. Frequency was indicated on a scale of 1 = never; 2 = less than once per week; 3 = once per week; 4 = once per day; and 5 = multiple times per day. A “not applicable” response option was also available. Presuming an ordinal scale of values for respondent ratings (as for the importance and satisfaction ratings), as appropriate, an ANOVA or robust ANOVA test was performed to determine any statistically significant differences between on- and off-campus students in the mean frequency of use reported for DSO functions. Figure 4 summarises the mean frequency of use ratings obtained, including estimated 95% confidence intervals. Based on a significance level of p < 0.01, where there was a significant difference in the mean frequency of use reported, this is indicated by a ‘*’ next to the DSO function. 1.0 1.5 2.0 2.5 3.0 3.5 4.0 1 2* 3 4* 5* 6 7 8 9 10* 11* 12* 13* 14* 15* 16 17 18 19* 20* M e a n f re q u e n cy r a ti n g DSO Function On-campus students Off-campus students Figure 4 Mean frequency of use ratings for DSO functions A key observation here is that for all cases where a significant difference was observed between on- and off-campus respondents, on-campus students reported the higher mean frequency of use of that DSO function. Although not directly comparable on a function-by-function basis, previous research based on responses from on- and off-campus engineering students at Deakin University indicated that off-campus students spent more hours per week online, were more regular users of email, and were more frequent users of online discussions than their on-campus counterparts—about double in all three cases (Palmer, 2001). This apparent rank reversal over the intervening decade seems unlikely to be the result of off-campus students actually reducing their frequency of use of online systems. Rather, it is more likely that on-campus students have significantly increased the frequency of their use of online systems due to the dramatic growth in the use of online systems in on-campus teaching during the same period (Smith & Caruso, 2010)—so called ‘blended learning’. So, while the previously observed large differences in the general use of online technologies may be now more equalised, in the case of certain DSO functions significant differences remain, with on-campus students reporting statistically significant higher mean frequency of use for more than half of the DSO functions surveyed. One potential impact of this observed difference is in the application of ‘academic analytics’. Academic analytics is the term applied to the mining of data from various university information systems for the purposes of improving decision making (Goldstein, 2005). It has been proposed that information about students’ interactions with the university could be used to predictively identify students at risk of not succeeding in their studies, and to take some action to intervene before matters become too serious. Frequency of Journal of Open, Flexible, and Distance Learning, 16(1) 79 student interaction with the university LMS is one of the claimed predictors of student performance (Dawson, McWilliam, & Tan, 2008; Morris, Finnegan, & Wu, 2005). However, more recent research in this area has highlighted the need for more investigation of the impact of student demographics (Fritz, 2011), including course delivery modality (Macfadyen & Dawson, 2010). It seems clear that systems premised/designed/templated on models of frequency of on- campus student interactions with an OLE should consider the possibility that off-campus students may interact significantly differently with some aspects of the system. For example, an academic performance ‘early warning’ system designed to detect students at risk on the basis of some measure(s) of student interactions with an OLE could potentially return false positive indications for off-campus students compared with on-campus students, if mode of study is not taken into account. Given that Deakin University is currently investigating the use of such academic analytics tools, and the fact that off-campus students make up nearly one-third of the student population, the observed differential frequency of access of DSO is a factor that should be taken into consideration. Overall satisfaction Respondents were asked to indicate their level agreement with three statements regarding their overall perceptions of DSO on a scale of 1–5 – 1 = strongly disagree; 3 = neutral; and 5 = strongly agree. Table 2 presents the statements, the mean rating responses for on- and off-campus students, and the result of an ANOVA test of the significance of the observed difference in the mean ratings. While mean ratings for off-campus students were higher than on-campus students for all three items, only one item had a significantly different rating – off-campus students had a significantly higher level of agreement with the statement ‘DSO is reliable and available when I need it’. It is not immediately clear why off-campus students would perceive the same system used by all students as more reliable. Table 2 Mean ratings of overall satisfaction with DSO Statement On-campus Off-campus Significance DSO enhances my learning 4.25 4.33 F1314 = 2.34 p > 0.12 DSO is reliable and available when I need it 3.93 4.21 F1314 = 23.85 p < 2x10 -6 I am satisfied with the level of support I received in using DSO in my studies 3.82 3.93 F1314 = 2.95 p > 0.08 Deakin University is in the process of transitioning to a new LMS system that will provide the core functionality for DSO. The final item on the DSO evaluation survey asked students whether they have been adequately informed about the process of changing the LMS. The overall “yes” response was 38.8%. Fisher’s exact 2 x 2 contingency test of response by mode of study was possible, and there was no significant difference in the response proportions between on- and off- campus students (p > 0.058). Conclusion This paper presents a large-scale, up-to-date and fine-grained investigation of the differences between on- and off-campus student engagement with an OLE, using an Australian university with a significant off-campus student enrolment base as a case study. It was found that, in some ways, on- and off-campus students are becoming more alike—the primary place of access to DSO for both groups was home, mobile access to DSO seems likely to be of growing importance Palmer, S. 80 to both groups, and there was no statistically significant difference in the mean satisfaction ratings between on- and off-campus students for virtually all DSO functions. Compared with research conducted more than a decade ago with engineering students at Deakin University which indicated that off-campus students accessed a range of online systems with about twice the frequency of on-campus students, the current investigation shows that frequency of access is now much closer, though not identical, for both groups. These observations are perhaps more evidence of a general trend in the blurring of the differences between on- and off-campus students—for example, in Australia a majority of on-campus, full-time enrolled students are engaged in significant part-time work (Devlin, James, & Grigg, 2008). While there are similarities between on- and off-campus student engagement with the OLE, there are still significant differences to be found in the details of the investigation. Off-campus students gave significantly higher mean ratings of importance (though not satisfaction) to a range of DSO functions that could be viewed as ‘value adders’ by off-campus students, enhancing their overall learning experience above and beyond what they would traditionally experience with printed study materials alone. The use of online quizzes stands out as the sole DSO function receiving significantly lower mean ratings from off-campus students for both importance and satisfaction, and further work is required to explore why this might be. For more than half of the DSO functions surveyed, on-campus students reported a statistically significantly higher mean frequency of access than off-campus students. This result may be important if an institution implements ‘academic analytics’-type systems that use frequency of OLE access as a trigger mechanism for identifying students at risk academically. More generally, the discovery of significant differences between on- and off-campus student responses to items on the DSO evaluation survey challenges the institution to understand the reasons for the observed differences and, if appropriate, act to ensure a level of equity in online experience for all students. The finding that elements of the institutional OLE are not universally perceived and used the same way by all student groups also challenges the value of standard, ‘one-size-fits-all’ institutional policies and templates relating to the use of OLEs. References Anderson, H. D. (2009). Formative assessment: Evaluating the effectiveness of online quizzes in a core business finance course. Journal of Distance Learning, 13(1), 26–40. Calvert, J. (2005). Distance education at the crossroads. Distance Education, 26(2), 227–238. Cook, C., Heath, F., & Thompson, R. L. (2000). A meta-analysis of response rates in web- or internet-based surveys. Educational and Psychological Measurement, 60(6), 821–836. Dawson, S., McWilliam, E., & Tan, J. P-L. (2008). Teaching smarter: How mining ICT data can inform and improve learning and teaching practice, 25th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (pp. 221–230). Melbourne, Australia: ASCILITE. Devlin, M., James, R., & Grigg, G. (2008). Studying and working: A national study of student finances and student engagement. Tertiary Education and Management, 14(2), 111–122. Fritz, J. (2011). Classroom walls that talk: Using online course activity data of successful students to raise self-awareness of underperforming peers. The Internet and Higher Education, 14(2), 89–97. Goldstein, P. J. (2005). Academic analytics: The uses of management information and technology in higher education. Boulder, CO: EDUCAUSE. Journal of Open, Flexible, and Distance Learning, 16(1) 81 Larreamendy-Joerns, J., & Leinhardt, G. (2006). Going the distance with online education. Review of Educational Research, 76(4), 567–605. Liu, C.-C., & Tsai, C.-C. (2008). An analysis of peer interaction patterns as discoursed by on-line small group problem-solving activity. Computers & Education, 50(3), 627–639. Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for educators: A proof of concept. Computers & Education, 54(2), 588-599. Martz, W. B., & Shepherd, M. M. (2007). Online internet testing: Lower perceived performance by students. In M. K. McCuddy, H. van den Bosch, W. B. Martz, A. V. Matveev, & K. O. Morse (Eds.), The challenges of educating people to lead in a challenging world (Vol. 10, pp. 499–511). Dordrecht, The Netherlands: Springer Netherlands. Morris, L. V., Finnegan, C., & Wu, S-S. (2005). Tracking student behavior, persistence, and achievement in online courses. The Internet and Higher Education, 8(3), 221–231. Palmer, S. (2001). On- and off-campus engineering student usage of a computer conferencing system. Journal of Research on Computing in Education, 33(3), 280–298. Palmer, S., & Holt, D. (2010). Students’ perceptions of the value of the elements of an online learning environment: Looking back in moving forward. Interactive Learning Environments, 18(2), 135–151. Reynolds, D., Treharne, D., & Tripp, H. (2003). ICT—the hopes and the reality. British Journal of Educational Technology, 34(2), 151–167. Saba, F. (2005). Critical issues in distance education: A report from the United States. Distance Education, 26(2), 255–272. Salinas, M. F. (2008). From Dewey to Gates: A model to integrate psychoeducational principles in the selection and use of instructional technology. Computers & Education, 50(3), 652–660. Smith, S. D., & Caruso, J. B. (2010). The ECAR study of undergraduate students and information technology, 2010. Boulder, CO: EDUCAUSE. Stafford, T. F. (2005). Understanding motivations for internet use in distance education. IEEE Transactions on Education, 48(2), 301–306. West, R., Waddoups, G., & Graham, C. (2007). Understanding the experiences of instructors as they adopt a course management system. Educational Technology Research and Development, 55(1), 1–26. Woods, R., Baker, J. D., & Hopper, D. (2004). Hybrid structures: Faculty use and perception of web-based courseware as a supplement to face-to-face instruction. The Internet and Higher Education, 7(4), 281–297. Xie, X., Lin, F., & Zhang, T. (2001). Comparison between on- and off-campus behaviour and adaptability in online learning: A case from China. Behaviour and Information Technology, 20(4), 281–291. Young, F. R. & Gibbings, P. (2007). Learning flexibility: The environment and a case study, 24th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (pp. 1121–1133). Singapore: ASCILITE. Palmer, S. 82 Biographical notes Dr Stuart Palmer spalm@deakin.edu.au Associate Professor Stuart Palmer’s research interests include the evidence-based evaluation of teaching and learning initiatives. � This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License. Palmer, S. (2012). Understanding the context of distance students: Differences in on- and off- campus engagement with an online learning environment. Journal of Open, Flexible and Distance Learning, 16(1), [pp. 70–82]. mailto:spalm@deakin.edu.au%3cmailto:spalm@deakin.edu.au http://creativecommons.org/licenses/by-nc-nd/3.0/ http://creativecommons.org/licenses/by-nc-nd/3.0/ http://creativecommons.org/licenses/by-nc-nd/3.0/