Journal of Student Affairs in Africa | Volume 3(2) 2015, 17-32 | 2307-6267 | DOI: 10.14426/jsaa.v3i2.133 www.jsaa.ac.za Faculty intervention as support for first-year students Ana’dhavelli Naidoo* and Juan-Claude Lemmens** Research article * Prof. Ana’dhavelli Naidoo is Deputy Director: Academic Support in the Department for Education Innovation, University of Pretoria, South Africa. Email: ana.naidoo@up.ac.za ** Dr Juan-Claude Lemmens is Head: Research and Innovation (Higher Education) in the Department for Education Innovation, University of Pretoria, South Africa. Email: jlemmens@up.ac.za Abstract The impetus for this study is grounded in a strategic decision by management to measure readiness for university education as part of an early alert and referral system. The motivation for this project is also rooted in literature that points out that the South African higher education system faces challenges with students entering the system underprepared. Data at entry to the university, specifically related to the individual student, is used initially to profile the students. This profile is used to identify students who could be at risk of failing. These students are referred to a Faculty Student Advisor (FSA) for support to address their needs. Using a survey, 966 students were identified as being at risk at the beginning of the 2013 academic year. After additional criteria were applied to our prediction model, 200 students were selected for academic development workshops or individual sessions provided as intervention in the first semester. An outcomes assessment method was used to determine whether the number of sessions that at-risk students attend has had an influence on their academic achievement in the first semester. The assumption is that students who made more use of the intervention services (attended more sessions) were more likely to be successful than students who defaulted on the intervention or attended fewer sessions with the FSAs. A cross-tabulation showed a significant association on the Pearson’s Chi-square statistic (13.60, df(4), p = 0.009), which implies that students who attend more sessions with the FSA are more likely to be academically successful in their first semester. Keywords Academic risk profile, academic success, first-year student, readiness for university. 18 Journal of Student Affairs in Africa | Volume 3(2) 2015, 17-32 | 2307-6267 | DOI: 10.14426/jsaa.v3i2.133 Background In 2010, a decision was made to provide a number of interventions to support students to succeed at the university. The aim was to develop an early alert and referral system to identify students who required support. One of the strategic goals of the university is to increase the access, throughput and diversity of students. Bringing the three concepts together results in a focus on the success of our diverse group of students. Unpublished cohort research on student success done at our institution over a number of years indicates that first-year students are especially at risk of failure and eventual withdrawal, which adversely impacts on the institution’s success indicators. The challenges faced by our university are not experienced in isolation. Literature (Van Zyl, 2013; Scott, 2009) shows that the South African higher education system as a whole is facing similar challenges. Since the publication of the White Paper on the higher education landscape (DoE, 1997) an updated report by the Council on Higher Education (CHE, 2014) shows that the key indicators of the success of the sector, such as participation and retention rates, are lower than anticipated. The gross participation rate in South Africa has, for instance, plateaued at about 17% (CHE, 2014). Graduation rates disaggregated by race show that, although African students have made some improvements in gross participation rates (improvement from 9% to 14% between 1996 and 2012), there is still a large disparity between their participation rates and their proportional population size. Proportionately, white and Indian students have much higher participation rates in the higher education system (CHE, 2009; 2013; 2014). Given that the number of students who gain access to the university is, relatively speaking, very low, it is of concern that only 51% of the students who gain access to three-year degree programmes at universities tend to graduate after period of six years (CHE, 2009; 2014; Bunting et al., 2010). In addition, these graduation rates are also highly skewed among racial groups (CHE, 2013). In order to increase students’ chances of success, higher education institutions need to make a concerted effort. Success, in this context, is defined as the ability of an institution to retain a student from admission through to graduation (Berger & Lyon, 2005). Research suggests that evaluating success only towards the end of a programme is insufficient and that students should be assessed and monitored from the very first day they enter the university (arguably even before then) and at strategic points along the cycle of the academic programme (Van der Merwe & Pina, 2008; Rassen et al., 2013; Van Zyl, Gravett & De Bruin, 2012; Van Zyl, 2013). Practices are constantly being put in place in the hope that they will influence student success. The thinking underlying this study is that improvement of students’ overall experience inside and outside the classroom, especially at first-year level, is a prerequisite for ensuring that students succeed in subsequent years of study. Scott (2009) writes that the first year is regarded as the academic year in which students’ success is highly influenced by their experiences. They have to adjust to the new institutional environment and manage increased levels of stress (Tinto, 1993; Bean & Eaton, 2000; Upcraft, Gardner & Barefoot, 2005; Jones et al., 2008; Hawkins & Larabee, 2009). Ana’dhavelli Naidoo & Juan-Claude Lemmens: Faculty intervention as support for f irst-year students 19 Literature review Internationally, institutions have approached some of the problems discussed above with an orchestrated approach by implementing an early alert, monitoring and referral system (Rassen et al., 2013; Tinto, 2013). Early alert refers to the identification of a student who is potentially at risk of being unsuccessful at a university, either academically or personally (Beck & Davidson, 2001; Seidman, 2005). Such a system is heavily focused during the first academic year, as numerous research points to the first-year learning experience as critical to student success and persistence (Reason, Terenzini & Domingo, 2006; Pascarella & Terenzini, 2005). According to Wend (2006), the student learning experience can be defined as the variety of experiences within the sphere of the university that influence learning. The student learning experience is therefore all-embracing and includes matters such as curricula; methods of teaching, learning and assessment; the learning environment and resources; student progress and achievement; and academic and pastoral support. The first- year experience is not only influenced by the university environment in which students go to class, socialise in cafeterias, participate in sport or learn in small groups in the library, but is also highly influenced by students’ motivations, ability, socio-economic status, preparedness, and other external factors (Tinto; 1993; Bean & Eaton, 2000; Braxton & Hirschy, 2005; Jones et al., 2008; Hawkins & Larabee, 2009). Particularly, for first-year academic achievement at the university, the level of academic readiness or preparedness is of critical importance. Literature suggests that a reason for the poor performance of the higher education sector is that students enter the system underprepared (Scott, Yeld & Hendry, 2007; Strydom as cited in Joubert, 2002). Academic readiness is broadly defined as the level of preparation a student needs in order to enrol and succeed, without remediation, in a credit-bearing programme at a higher education institution (Conley, 2007). More specifically, preparedness refers to being prepared in reading, writing and mathematical skills (Van Dyk & Weideman, 2004; Cliff, 2014). The readiness model of Conley (2007) shows that readiness for university education is not only associated with academic performance at school or with measures of ability on psychometric tests, but also with socio-cultural and motivational factors. The participants of Byrd and MacDonald’s study, for instance, identified the following additional factors associated with readiness, namely: skills in time-management; motivational factors; background factors; and student self-concept (Byrd & MacDonald, 2005). Other researchers pay attention to the non-cognitive and/or demographic characteristics of students as influencers of readiness for university (Sedlacek, 2004, 2005; Camara, 2005a, 2005b). The four categories of readiness that were identified by Byrd and MacDonald’s (2005) qualitative study are confirmed by Conley’s (2007) research on readiness for university over a number of years. Conley further suggests a broad definition of readiness that includes cognitive strategies, acquiring content knowledge, academic behaviours, and contextual knowledge and skills. Conley (2007) explains that the various elements of readiness are neither mutually exclusive nor perfectly nested because they interact with and affect one another extensively. Entry characteristics in the form of demographic variables have been shown to predict accomplishment later in one’s academic career (Sedlacek, 2005). 20 Journal of Student Affairs in Africa | Volume 3(2) 2015, 17-32 | 2307-6267 | DOI: 10.14426/jsaa.v3i2.133 Keup (2008) names four issues that impact on student readiness: the shift to a truly multicultural student body; mental and emotional healthcare needs; a utilitarian view of higher education; and an integration of new technologies. In considering the first-year experience, Kift (2009) says that the challenge of moving from research and theory into practice is becoming more difficult. This makes it even more important to determine whether our practices will have any impact on student success. The interventions provided by the FSAs can be classified as informal learning opportunities (Wawrzynski and Baldwin, 2014) that also forge links with the formal ones taking place in the classroom. While interventions by the FSAs may not fall neatly into Kuh’s (2008) high-impact practices, they are seen as an attempt to address some of the needs of first-time-entering students. Early alert and referral strategy Our institution adopted an early alert and referral system early in 2010 to facilitate the transition from school to university in order to have students fully integrated into the university environment within the first quarter of the first year. Among the activities were academic introductions to the disciplines before the start of the official year, allocation of mentors to particular students, monitoring student achievement after the first test (in a particular faculty), and the placement of advisors to provide support within each faculty. In line with literature on student readiness (Van der Merwe & Pina, 2008; Van Zyl, Gravett & De Bruin, 2012; Rassen et al., 2013; Van Zyl, 2013), students are assessed on their academic readiness. Our Student Academic Readiness Survey (STARS) was developed to function – in conjunction with demographic variables, high-school marks (Admission Point Score) and the National Benchmark Test – as early warning indicators of failure or dropout among first-year students. The STARS is a low-stakes, self-report survey measuring non-cognitive variables on 115 items, administered during the orientation week. This survey has been administered, since 2010, to over 42 000 students. It is a norm- referenced test and consists of 26 non-cognitive dimensions. The objectives of the STARS are to act as early-warning indicators of failure or dropout among first-year students and to categorise students into groups for specific interventions. Students identified through the STARS are referred to a peer mentorship programme for transitional support and/or an FSA for academic support and advice. The FSA refers students with financial and accommodation challenges to the relevant sections. This study will focus on the academic development interventions supplied by the FSAs. The responsibilities of the FSAs are to make contact with students and invite them to an intervention programme; advise such students about reducing their risks; provide study skills and time management workshops; monitor particular students’ results as an early alert of their progress; provide support to self-referred students; and assist students requiring advice about programme changes. The FSAs are also required to keep records of the students who attend their intervention programmes. The data about the number of students identified for the intervention programmes as well as the students’ participation rates are necessary for the programme to have value during the first semester of the first year. Research suggests that Ana’dhavelli Naidoo & Juan-Claude Lemmens: Faculty intervention as support for f irst-year students 21 not only student support services, but the system as a whole, influence students’ learning experiences and success (Tinto, 2013). While efforts are also being made to encourage lecturers to support students within their discipline, this is not the focus of this article. Here, we focus on the interventions made by the FSAs. Methodology The current study focuses on the quantitative analysis of survey data in combination with high-school academic results, student demographic data and results from the National Benchmark Test (NBT). The aforementioned data are used in predictive analytics to identify students for various intervention programmes on campus. Data collection method or procedure The STARS is administered to students attending the orientation week at the beginning of each academic year. The survey is intended for all first-time-entering, first-year students. However, not all new students are able to attend the orientation week and, in some cases, returning and transferring students also attend it. The surveys are administered in paper- and-pencil format and electronically. The results of the STARS feed directly into the institution’s business intelligence software, called the STARS Student Retention System. Each student’s STARS profile is programmatically compared with his or her high school academic results, student demographic data and results from the NBT. Predictive analytics of the data over a number of years were used to develop academic risk profiles of students. The STARS retention system uses the algorithms of the predictive analytics to identify the students who may be at risk. The system produces automatic reports that are used by the FSAs to contact students about a variety of intervention programmes. The criteria for selection for academic advising, which are evaluated in this study, include high school academic performance in relation to the admission requirements per programme. As part of this study we have included the NBT as an additional criterion for selecting at-risk students because research shows that the three NBT sub-tests generally act as contributing signals, with National Senior Certificate (NSC) results, in explaining first- year academic achievement (Lemmens, 2013). The results of the STARS are mainly used as qualitative information for tailoring the intervention programme. FSAs are required to keep records of the students who attend academic advising and the number of sessions they attend. The number of sessions ranged from zero to nine sessions. These sessions were clustered into three, roughly even, groups in order to perform a Pearson’s Chi-square analysis from contingency tables (Field, 2005). The students who attended zero sessions were contacted but chose not to attend any of the sessions. Three distinct groups – at-risk, borderline and successful – were used by the FSAs for further academic development interventions in the second semester. The data for this is not presented here as the focus is on the first semester only. The research question for this study is: Does the number of academic advising sessions decrease the academic risk of students who were predicted to be at risk upon entry to the institution? 22 Journal of Student Affairs in Africa | Volume 3(2) 2015, 17-32 | 2307-6267 | DOI: 10.14426/jsaa.v3i2.133 Data analysis The data analysis for this study can be categorised as descriptive analysis of the demographic data and institutional sample data. Inferential statistics, namely a Pearson’s Chi-square, was used in combination with contingency tables because the data were clustered into discrete categories. A Chi-square analysis was also used to test a hypothesis with an associated significance indication (Field, 2005). In order to evaluate the academic success of students, a cluster analysis was performed to identify students who had performed academically poorly in the first semester. Cluster analysis is a statistical method for finding relatively homogeneous clusters of cases based on measured characteristics. The k-means algorithm clustering method was utilised to analyse the data (Field, 2005). The variables that were used in the first phase of the analysis, performed by our Institutional Planning section, were the ratio of credits registered versus credits failed; the average mark for the first semester; the ratio of modules in which students performed poorly; and the high school English mark. In the second round of the analysis, the borderline students were extracted from the data set and split into Sciences and non-Sciences groups. The k-means cluster analysis was performed on these two groups separately. The variables that were used in the second phase were the ratio of credits registered versus credits failed; the average mark for the first semester; and the ratio of modules in which students performed poorly. However, the high school English mark was removed. This allowed identification of borderline students at the granular level, thus splitting the lower borderline from the true borderline and upper borderline students. The lower borderline students became part of the at-risk group. The following outcome assessment models will be used to evaluate the outcomes as they are presented in the evaluation framework: Model 1: Outcomes assessment of at-risk students Characteristics of at-risk students at entry Academic development Transition Financial and accommodation Academic outcomes Fail first semester Dropout/course change Academic risk cluster Participation rates Successfully complete intervention Discontinue intervention Ana’dhavelli Naidoo & Juan-Claude Lemmens: Faculty intervention as support for f irst-year students 23 In this model, the students who were identified as being at risk with predictive analytics were referred to one of the three support services mentioned above. They are subsequently prearranged in two groups, namely students who successfully completed the intervention and students who discontinued or defaulted on the intervention. The two groups will be compared in relation to their first-semester academic achievement, their academic risk cluster and dropout rate. The purpose is to determine whether students who made use of the intervention services were more likely to be successful than the students who defaulted on the intervention. In this study the focus is on the cluster analysis of the academic outcomes in relation to participation in academic development sessions facilitated by the FSAs. Sample The population for this study consists of 12 916 students enrolled in their first year in 2013. First-time-entering, first-year students numbered 8 515, with the remainder of the students being both returning students (students who did not successfully progress to the second year of study) and transferring students (students who changed course or transferred from other institutions). A total of 7 033 students completed the STARS and, from the initial risk criteria, 966 students were identified as being at risk. After applying additional risk criteria explained in the data collection procedure above, 200 students were identified as being at risk. The FSAs followed these students up. The FSAs kept records of the 200 students for the purpose of this study. Coyne (1997) refers to all sampling as being purposeful. He turns to Patton’s view (1990, p. 69), that “qualitative inquiry typically focuses in depth on relatively small samples, even single cases, selected purposefully”. The purpose of choosing the 200 students is that they interacted with the FSAs in terms of being provided with a form of intervention. Ethical considerations Confidentiality of first-year students was maintained at all costs. Students were informed of the purpose of the survey prior to its administration. Students had to log in to the student portal with their student numbers and passwords to gain access to their individual reports. Only reports of students who agreed to make their information accessible to university staff were available by proxy access to FSAs, counselling staff and deans of faculty. Students were advised to make use of support structures, but not forced to do so. These students were also briefed about the method through which they had been identified and what the support programme entailed. Students who decided not to make use of recommended services either did not attend the sessions or indicated their decision to default verbally to the FSA when invited to the intervention programme. Results Our results focus on the descriptive statistics of the sample and the quantitative evaluation of the intervention programme facilitated by the FSAs. 24 Journal of Student Affairs in Africa | Volume 3(2) 2015, 17-32 | 2307-6267 | DOI: 10.14426/jsaa.v3i2.133 Sample The distribution of first-year students by faculty and by admittance type can be observed in Table 1. Table 1: Biographical data of first-year students by faculty and admission type Faculty New Returning Transfer Total Humanities 1 394 244 80 1 718 Natural and Agricultural Sciences 1 716 663 81 2 460 Law 173 180 9 362 Theology 56 11 7 74 Economic and Management Sciences 1 717 383 95 2 195 Veterinary Science 131 42 27 200 Education 914 98 54 1 066 Health Sciences 569 646 651 1 866 Engineering, Built Environment and IT 1 845 1 085 45 2 975 Total 8 515 3 352 1 049 12 916 66% 26% 8% 100% From the population of first-year students admitted to the university (12 916), a total of 7 033 students completed the STARS. Of the 7 033 students completing the STARS during the orientation programme, 93% are first-time-entering, first-year students, 2% of the registered students are students returning to the first year and 4% of the registered students have transferred from other institutions. One per cent of the students are labelled as “unknown” on their student record. The target audience for the student academic readiness survey was first-time-entering, first-year students and, to a large extent, this was achieved. The distribution in participation rates in the STARS by faculty is equivalent to the distribution of the student population as presented in Table 2, thus indicating that the students who completed the STARS are a representative sample of the first-time-entering, first-year student population. From Table 3 it is evident that the majority of students who completed the STARS are female, which is in accordance with the gender profile of the undergraduate student population (female = 58.5% and male = 41.5%) and first-year students in particular (female = 58.2% and male = 41.8%) at our own institution. Ana’dhavelli Naidoo & Juan-Claude Lemmens: Faculty intervention as support for f irst-year students 25 Table 2: Distribution of students completing the STARS by faculty Faculty Frequency Percentage Economic and Management Sciences 1 374 19.5 Education 834 11.9 Engineering, Built Environment and IT 1 550 22.0 Health Sciences 469 6.7 Humanities 986 14.0 Law 164 2.3 Natural and Agricultural Sciences 1 500 21.3 Theology 36 0.5 Veterinary Science 120 1.7 Total 7 033 100.0 Table 3: Distribution of students completing the STARS by gender Gender Frequency Percentage Female 4 120 58.6 Male 2 913 41.4 Total 7 033 100.0 Table 4: Distribution of students completing the STARS by race Racial group Frequency Percentage Asian 363 5.2 Black 2 705 38.5 Coloured 178 2.5 Other 34 0.5 Unknown 6 0.1 White 3 747 53.3 Total 7 033 100.0 From Table 4 it is evident that the majority of students who completed the STARS are categorised as white, which is roughly in accordance with the racial profile of the undergraduate student population and first-year students in particular. Black students make up 38.5% of the sample, while the proportion of black students in the population of first- time-entering, first-year students is 41.1%. The proportion of white students completing the STARS is 53.3%, while the population of white, first-time-entering, first-year students is 50.7%. Black students are thus slightly under-represented in the sample and white students are slightly over-represented in the sample. 26 Journal of Student Affairs in Africa | Volume 3(2) 2015, 17-32 | 2307-6267 | DOI: 10.14426/jsaa.v3i2.133 Academic proficiency of students completing the STARS The academic proficiency of students who completed the STARS is presented in Table 5. Academic proficiency, in this instance, is measured by the Admission Point Score (APS) as well as the subtests of the National Benchmark Test (NBT). These scores show, to some extent, the academic skills and/or knowledge that a student has acquired up to a certain point. Universities can use this information to set benchmarks for the level of knowledge and/or skills that students require to have a fair chance of being successful at a particular programme. Thus, they not only say something about the students’ current ability, but also predict academic outcomes in the future based on what is known presently. Table 5: Average high-school academic performance on the APS and NBT N Min Max Mean Std deviation APS 6 574 15 50 34.50 4.589 NBT-AL 5 139 28 92 62.64 11.311 NBT-QL 5 139 13 96 58.88 15.128 NBT-Math 4 510 19 97 48.84 15.947 Sample total 4 276 From Table 5 the mean APS score was 34.5. On the other hand, the mean score for the NBT–Academic literacy subtest was 62.64, while the mean score for the NBT– Quantitative literacy subtest was 58.88 and the mean score for the NBT–Mathematics subtest was 48.84. The sample total for this table is 4 276 because not all students have to complete the NBT results, and the sample total represents the number of students who have scores for all four of the variables. First-semester students ‘at risk’ according to the STARS Table 6 shows that 966 students were identified as being at risk by the STARS Student Retention System at the beginning of the academic year, using only APS as the criteria. For the purpose of this study, the NBT was included in the selection criteria to identify students who are at a greater risk. A total of 200 students were selected for this purpose. Internal research at our institution on NSC subjects and APSs with the STARS results shows that the NSC – and, more specifically, the APS – only partly explains academic outcomes (Lemmens, 2013). When other variables, such as NBT results and psychosocial variables, are added, they can increase the accuracy of the prediction model. From Table 7 it is evident that students who were selected based on APS criteria only have higher mean scores on the NBT-AL, NBT-QL and NBT-Math than the students selected with APS and NBT criteria (60.64 vs 56.23; 57.24 vs 49.91; 45.27 vs 39.05 respectively). One can see that the APS scores of both groups were almost equal and that the addition of the NBT subtest provides additional information to help with the accuracy of the prediction model. The students who were contacted were prioritised because of their performance on the NBT and would possibly be at greater academic risk. Ana’dhavelli Naidoo & Juan-Claude Lemmens: Faculty intervention as support for f irst-year students 27 Table 6: Prediction criteria applied Faculty Criteria: APS Criteria: APS and NBT Economic and Management Sciences 264 42 Education 97 23 Engineering, Built Environment and IT 215 13 Health 33 2 Humanities 242 43 Law 12 3 Natural and Agricultural Studies 83 65 Theology 7 0 Veterinary Science 13 9 Grand total 966 200 Table 7: Descriptive statistics of participation in FSA session/s and proficiency scores APS criteria only (N = 766) N Min Max Mean Std deviation APS 763 17 38 29.83 3.679 NBT-AL 519 29 85 60.64 11.319 NBT-QL 519 20 91 57.24 15.981 NBT-Math 439 20 88 45.27 14.396 APS and NBT criteria (N = 200) N Min Max Mean Std deviation APS 196 3 37 29.09 3.523 NBT-AL 125 31 83 56.23 12.363 NBT-QL 125 19 84 49.91 14.011 NBT-Math 112 22 71 39.05 11.939 Academic development outcomes assessment In this section, the students who were identified as being at risk based on academic criteria will be assessed against the academic risk cluster as the outcome variable. The students who were identified and referred to the FSAs will be divided into three groups, analysed according to the number of sessions in which they participated (grouped for analysis purposes). 28 Journal of Student Affairs in Africa | Volume 3(2) 2015, 17-32 | 2307-6267 | DOI: 10.14426/jsaa.v3i2.133 Table 8: Cross-tabulation of academic risk cluster and number of FSA sessions participated in Academic cluster Number of individual FSA sessions Total Zero sessions One session Two or more sessions At risk Count 28 27 12 67 % within cluster 41.8% 40.3% 17.9% 100% Borderline Count 24 29 22 75 % within cluster 32.0% 38.7% 29.3% 100% Successful Count 8 28 22 58 % within cluster 13.8% 48.3% 37.9% 100% Total Count 60 84 56 200 The two variables in the contingency table (cross-tabulation) in Table 8 – academic risk cluster and the number of FSA sessions participated in – are significantly associated on the Pearson’s Chi-square statistic (13.60, df(4), p = 0.009). This implies that there is a significant relationship between the risk cluster in which a student is observed and the number of academic advising sessions in which a student participated with the FSAs. The category of “Zero sessions” refers to the students who were invited to the interventions but did not participate (n = 60). The number of students attending one session were 84 and number of students attending two or more sessions were 56. The results show that 41.8% of the at-risk students fall into the zero session, 40.3% participated in one session and 17.9% participated in two or more sessions. The students classified as borderline with the cluster analysis had slightly larger numbers of students attending one session. Namely, 38.7% of the students attended one session, 32% of the borderline students attend zero sessions and 29.3% attended two or more sessions. In contrast, 13.8% of the successful students attended zero sessions, 48.3% attended one session and 37.9% of the students attended two or more sessions. Based on the cluster analysis percentages, one could conclude that the optimal number of sessions that at-risk students should attend, to move from being classified as being at risk upon entry into the institution to being classified as successful, is one. One can also observe a larger number of students within the “Successful” category. However, students attending one session were in the majority, which will influence the relative frequencies and associated percentages. In order to accommodate the difference in total frequencies of students attending interventions, the relative percentages of the number of sessions in relation to the academic cluster has to be investigated. The results of the calculation of the frequencies and column totals in Table 8 (percentages not shown here but explained in this paragraph) show that successful students were more likely to have attended two or more sessions (39.3%) than students who attended one session (33.3%) or no sessions at all (13.3%). Conversely, the at-risk students tended not to attend any sessions (46.7%) compared with students who participated in one session (32.1%) and students who participated in two or more sessions (21.4%). The Ana’dhavelli Naidoo & Juan-Claude Lemmens: Faculty intervention as support for f irst-year students 29 students classified as borderline could have attended zero (40%), one (34.5%), or two or more (39.3%) sessions without showing a clear trend in the contribution of attending more sessions to risk movement. Generally, the results thus show that of the high-risk group contacted by the FSAs, students who attend more sessions with the advisor are more likely to have successful first-semester academic outcomes. Furthermore, there is evidence to show that academic advice is an effective intervention for high-risk students in the short term. However, this finding needs to be tested further. Further research will be undertaken on students who did not receive any intervention, but who were identified by the STARS, to determine if they could have been more successful had they received academic advice. Such a comparison could have worked well with the students whom the STARS system identified as not being at risk. Conclusion The research from this study found that 766 STARS students who were selected using APS criteria and who did not attend FSA interventions were less likely to be at risk than the 200 who were selected based on the APS and NBT results participated. Adding the subtest of the NBT to the identification of students for the academic intervention programmes has improved the ability to predict the success of students entering the university, allowing the university to be more active in selecting at-risk students with predictive analytics and recommending interventions proactively. An outcomes assessment of this analysis– intervention–evaluation framework has shown promising results for the implementation of academic advising in this case, and has allowed us to improve the intervention programme as well as the monitoring of at-risk students. The results show that only 32 out of 200 students were able to move out of academic risk without attending any academic advising – however, most of these students were at borderline academic achievement. This could be attributed to students making changes to their academic behaviour due to the knowledge that they were being observed (the Hawthorne effect) or because they consulted elsewhere. The results show that 28 of the 60 students who did not attend any sessions are from the at-risk category (46.7%). It is also evident that students who attend only one session have a one in three chance of being either at risk, borderline or successful. This indicates that attending only one session does not seem to be clearly beneficial to this group of students. Evidently, the tipping point in the effectiveness of academic advising is student participation in two or more sessions. Students who attend two or more sessions are likely to be successful or, at least, to become borderline academic achievers. Clearly, students who do not use FSA services are at a higher risk. Thus, active participation in intervention programmes is of value to academically underprepared students. Through the STARS and the employment of FSAs, the institution has shown its commitment to interventions for improving student success. It is taken further within the faculties where students identified as being at risk are supported by tutors and other academic practices. The findings have also shown the value of using NBT results as additional criteria for identifying at-risk students for academic advice programmes. Research has used NBT 30 Journal of Student Affairs in Africa | Volume 3(2) 2015, 17-32 | 2307-6267 | DOI: 10.14426/jsaa.v3i2.133 mostly in comparative analyses of NSC marks and NBTs for placement purposes (Fleisch, Schöer & Cliff, in press). References Bean, J.P. & Eaton, S.B. (2000). A Psychological Model of College Student Perception. In J.M. Braxton (Ed.). Reworking the Student Departure Puzzle (pp. 48–61). Nashville, TN: Vanderbilt University Press. Beck, H.P. & Davidson, W.D. (2001). Establishing an Early Warning System: Predicting Low Grades in College Students from Survey of Academic Orientations Scores. Research in Higher Education, 42(6), 709–723. Berger, J.B. & Lyon, S.C. (2005). Past to Present: A Historical Look at Retention. In A. Seidman (Ed.). College Student Retention: Formula for Student Success (pp. 1–29). Westport: Praeger Publishers. Braxton, J.M. & Hirschy, A.S. (2005). Theoretical Developments in the Study of College Student Departure. In A. Seidman (Ed.). College Student Retention: Formula for student success (pp. 61–87). Westport: Praeger Publishers. Bunting, I. (2006a). The Higher Education Landscape Under Apartheid. In N. Cloete, P. Maassen, R. Fehnel, T. Moja, T. Gibbon & H. Perold (Eds.). Transformation in Higher Education: Global Pressures and Local Realities (pp. 35–52). Dordrecht: Springer. Bunting, I. (2006b). Students. In N. Cloete, P. Maassen, R. Fehnel, T. Moja, T. Gibbon & H. Perold (Eds.). Transformation in Higher Education: Global Pressures and Local Realities (pp. 95–111). Dordrecht: Springer. Bunting, I., Sheppard, C., Cloete, N. & Belding, L. (2010). Performance Indicators: South African Higher Education 2000–2008. Cape Town: CHET. Retrieved 14 June 2014 from http://chet.org.za/ books/performance-indicators-south-african-higher-education-2000-E2-80-932008 Byrd, K., & MacDonald, G. (2005). Defining College Readiness from the Inside Out: First Generation College-student Perspectives. Community College Review, (33)1, 22–37. Camara, W.J. (2005a). Broadening Criteria of College Success and the Impact of Cognitive Predictors. In W.J. Camara & E.W. Kimmel (Eds.). Choosing Students: Higher Education Admissions Tools for the 21st Century (pp. 53–79). New Jersey: Lawrence Erlbaum Associates. Camara, W.J. (2005b). Broadening Predictors of College Success. In W.J. Camara & E.W. Kimmel (Eds.). Choosing Students: Higher Education Admissions Tools for the 21st Century (pp. 81–105). New Jersey: Lawrence Erlbaum Associates. Cliff, A. (2014). Entry-level students’ reading abilities and what these abilities might mean for academic readiness. Language Matters, 45(3), 313–324. Cliff, A. (2015). The National Benchmark Test in Academic Literacy: How might it be used to support teaching in higher education? Language Matters, 46(1), in press. Conley, D.T. (2007). Redefining College Readiness (Vol. 3). Center for Educational Policy Research: University of Oregon. Centre for Higher Education Transformation. (n.d.). Retrieved 20 June 2013 from http://chet.org.za/ data/sahe-open-data#keydata Council on Higher Education (CHE). (2009). Higher Education Monitor: The State of Higher Education in South Africa. Pretoria: Council on Higher Education. Council on Higher Education (CHE). (2013). A proposal for undergraduate curriculum reform in South Africa: The case for a flexible curriculum structure. Pretoria: Council on Higher Education. Council on Higher Education (CHE). (2014). Vital Stats: Public Higher Education 2011. Retrieved Ana’dhavelli Naidoo & Juan-Claude Lemmens: Faculty intervention as support for f irst-year students 31 November 18, 2014 from http://www.che.ac.za/media_and_publications/monitoring-and- evaluation/vitalstats-public-higher-education-2011 Coyne, I.T. (1997). Sampling in qualitative research. Purposeful and theoretical sampling; merging or clear boundaries? Journal of Advanced Nursing, 26(3), 623–630. Department of Education (DoE). (1997). Education White Paper 3. A Programme for the Transformation of Higher Education. General Notice 1196 of 1997. Pretoria: Department of Education. Field, A.P. (2005). Discovering Statistics using SPSS: (and sex and drugs and rock ’n’ roll). London: Sage Publications. Fleisch, B., Schöer, V. & Cliff, A. (In press). When signals are lost in aggregation: A comparison of language marks and competencies of entering university students. South African Journal of Higher Education.  Hawkins, V.M. & Larabee, H.J. (2009). Engaging Racial/Ethnic Minority Students in Out-of- class Activities on Predominantly White Campuses. In S.R. Harper & J. Quaye (Eds). Student Engagement in Higher Education: Theoretical Perspectives and Practical Approaches for Diverse Populations (pp. 179–188). New York: Routledge. Jones, B., Coetzee, G., Baily, T. & Wickham, S. (2008). Factors that Facilitate Success for Disadvantaged Higher Education Students. Rural Education Access Programme (REAP). Unpublished document. Joubert, J.M. (2002). ‘n Teoretiese Model vir die Toepassing Van Self-Gereguleerde Leer met Metakognitiewe Betrokkenheid as ‘n Tweede-orde Proses. Doctoral Thesis, University of Pretoria, South Africa. Keup, J.R. (2008), New challenges in working with traditional-aged college students. New Directions for Higher Education, 2008(144), 27–37. doi: 10.1002/he.323. Kift, S. (2009). A Transition Pedagogy: The First Year Experience Curriculum Design Symposium 2009. Higher Education Research and Development Society of Australasia Inc., 31(1), 2009. Kuh, G.D. (2008). High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter. Washington, DC: Association of American Colleges and Universities. Lemmens, J. (2013). Exploratory research into the prediction value of first semester academic performance on students’ progression to the second year of their course. Department for Education Innovation, University of Pretoria. Marshall, M.N. (1996). Sampling for qualitative research. Family Practice, 13(6), 522–525. Pascarella, T.E. & Terenzini, P.T. (2005). How College Affects Students: A Third Decade of Research (Vol. 2). San Francisco: Jossey-Bass. Patton, M.Q. (1990). Qualitative evaluation and research methods (2nd ed.). Thousand Oaks, CA, US: Sage. Rassen, P.R., Chaplot, P., Jenkins, D. & Johnstone, R. (2013). Understanding the Student Experience through the Loss/Momentum Framework: Clearing the Path to Completion. Berkeley, CA: The Research and Planning Group for California Community Colleges. Retrieved 19 August 2014 from http://www.rpgroup.org/sites/default/files/CbD-Understanding.pdf Reason, R.D., Terenzini, P.T. & Domingo, R.J. (2006). First Things First: Developing Academic Competence in the First Year of College. Research in Higher Education 42(2), 149-175. Scott, I. (2009). First-year Experience as Terrain of Failure or Platform for Development? In B. Liebowitz, A. Van der Merwe & S. Van Schalkwyk (Eds.). Focus on First-year Success: Perspectives Emerging from South Africa and Beyond (pp. 17–36). Stellenbosch: AFRICAN SUN MeDIA. Scott, I., Yeld, N. & Hendry, J. (2007). Higher Education Monitor 6: A Case for Improving Teaching and Learning in South African Higher Education. Pretoria: Council on Higher Education. 32 Journal of Student Affairs in Africa | Volume 3(2) 2015, 17-32 | 2307-6267 | DOI: 10.14426/jsaa.v3i2.133 Sedlacek, W.E. (2004). Beyond the Big Test: Noncognitive Assessment in Higher Education. San Francisco: Jossey-Bass. Sedlacek, W.E. (2005). The Case for Noncognitive Measures. In W.J. Camara & E.W. Kimmel (Eds). Choosing Students: Higher Education Admissions Tools for the 21st Century (pp. 177–193). New Jersey: Lawrence Erlbaum Associates. Seidman, A. (Ed.). (2005). College Student Retention: Formula for Student Success. Westport: Praeger Publishers. Tinto, V. (1993). Leaving College: Rethinking the Causes and Cures of Student Attrition. Chicago: University of Chicago Press. Tinto, V. 2012. Enhancing student success: Taking the classroom success seriously. The International Journal of the First Year in Higher Education, 3(1), 1–8. Retrieved November 24, 2013 from https:// fyhejournal.com/article/view/119/120 Tinto, V. 2013. Theoretical underpinnings and research framework for student success. Presentation to the CHE symposium, 20 August 2013, Pretoria. Retrieved November 21, 2014 from http://www.che. ac.za/content/regional-symposia-student-success-19-23-august-2013 Upcraft, M.L., Gardner, J.N. & Barefoot, B.O. (2005). Challenging and Supporting the First-year Student: A Handbook for Improving the First Year of College. San Francisco: Jossey-Bass. Van der Merwe, A. & Pina, R. (2008). Use of Portals, CRM, Assessment: Stellenbosch University Case Study. Workshop presented at the 1st Southern African Conference on the First-Year Experience: Using ICT to Track, Profile and Individually Help Students, Stellenbosch. Van Dyk, T. & Weideman, A. (2004). Switching Constructs: on the Selection of an Appropriate Blueprint for Academic Literacy Assessment. Journal for Language Teaching, 38(1), 1–12. Van Heerden, E. (1997). University education and African thought: Ref lections on underachievement among some UNISA students. South African Journal of Ethnology, 20(2), 76–94. Van Zyl, A. (2013). Teaching the students we have: Two perspectives on first year students at the University of Johannesburg and the UJ first year experience initiative. Paper presented at Higher Education Learning and Teaching Association of South Africa (HELTASA), Pretoria, South Africa. Retrieved November 21. 2014 from http://hdl.handle.net/10210/12491 Van Zyl, A., Gravett, S. & De Bruin, G.P. (2012). To what extent do pre-entry attributes predict first year student academic performance in the South African context? South African Journal of Higher Education, 26(5), 1095–1111. Wawrzynski, M. & Baldwin, R. (2014). Promoting high-impact student learning: Connecting key components of the collegiate experience. New Directions for Higher Education, 165, 51–62. Wend, P. (2006). Towards a joined-up student learning experience strategy at Oxford Brookes University. Brookes eJournal of Learning and Teaching 2.1. Retrieved 20 August 2014 from http:// bejlt.brookes.ac.uk/articles/towards_a_ joined_up_student_learning_experience_at_oxford_ brookes_universit/.