143 Research Article 2022 40(2): 143-158 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 Published by the UFS http://journals.ufs.ac.za/index.php/pie © Creative Commons With Attribution (CC-BY) Measuring risks associated with students of introductory statistics: Scale development and implementation Abstract A lack of understanding of statistics, and being unable to apply its techniques, not only has a negative impact on the pass rates of students studying introductory statistics, but also affects the potential usefulness of these students when they enter the marketplace. I report here a novel scale developed to measure various factors that may be associated with the risk of failing an introductory statistics module at university. Various scales are available, but a single scale succinctly combining the factors was not available. The risks students face were explored by creating a 17-item instrument, named the Statistics Students’ Risk Questionnaire (SSRQ). The resulting SSRQ scale consists of four dimensions (motivation, attitude towards statistics, statistical anxiety and academic procrastination) that were identified through exploratory factor analysis. The model was confirmed using confirmatory factor analysis. The reliability of the scale was determined, and its usefulness was demonstrated to identify at-risk students. The scale can be applied to determine overall risk but can also be used to reveal which factors contribute to the risk. The scale can be supplemented by including the mathematical history of students of concern as an additional risk factor. This enables the statistics lecturer to select an appropriate support plan to assist at- risk students based on which factors are flagged as risks. Keywords: Statistics students at risk, statistical anxiety, attitude towards statistics, motivation, academic procrastination, mathematical history, scale development. 1. Introduction Statistics is a crucial discipline for students, irrespective of what they are studying, since it is needed not only to evaluate empirical evidence but also to reinforce empirical research through the application of statistical techniques (Ridgway, Nicholson & McCusker, 2007). High failure rates in introductory statistics modules lead to increased costs for the students and the institution as well as to undergraduates being delayed entering their careers (Gultice, Witham & Kallmeyer, 2015). Since students often struggle with their statistics course, they develop negative attitudes towards AUTHOR: Ms Wilma Coetzee1 AFFILIATION: 1North-West University DOI: http://dx.doi. org/10.18820/2519593X/pie. v40.i2.11 e-ISSN 2519-593X Perspectives in Education 2022 40(2): 143-158 PUBLISHED: 08 June 2022 RECEIVED: 27 September 2021 ACCEPTED: 07 January 2022 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 http://www.statssa.gov.za/?p=11341 http://documents.worldbank.org/curated/en/530481521735906534/Overcoming-Poverty-and-Inequality-in-South-Africa-An-Assessment-of-Drivers-Constraints-and-Opportunities http://documents.worldbank.org/curated/en/530481521735906534/Overcoming-Poverty-and-Inequality-in-South-Africa-An-Assessment-of-Drivers-Constraints-and-Opportunities http://documents.worldbank.org/curated/en/530481521735906534/Overcoming-Poverty-and-Inequality-in-South-Africa-An-Assessment-of-Drivers-Constraints-and-Opportunities http://orcid.org/0000-0002-7418-4581 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 1442022 40(2): 144-158 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 Perspectives in Education 2022: 40(2) it (León-Mantero et al., 2020). The effectiveness of teaching statistical courses is therefore critical (Ramirez, Schau & Emmioglu, 2012). The purpose of this study was to develop a scale that can be used as an efficient detection tool to identify students who may be at risk of failing an introductory statistics module. Various scales exist, but currently no single scale measures the statistical anxiety, the attitude towards statistics, the tendency to procrastinate when faced with statistical assignments and the motivation of a student. The scale presented in this study measures all these factors in one concise questionnaire. This scale can be applied as a screening instrument to identify potentially at-risk undergraduates early in a semester, after which they should receive appropriate assistance. This is a better strategy than waiting for test results to identify those at-risk by their subsequent failure, since by then valuable time has been lost and foundational knowledge for the module has not been mastered (Gultice, Witham & Kallmeyer, 2015). It also provides valuable information on what contributes to the risk, making it easier to determine an appropriate course of action for someone identified as weak. This scale’s ability to identify at-risk students can be compared to existing scales. Universities can then decide if they prefer using only this scale in future, or in combination with other available scales. 2. Literature review Numerous studies have been conducted to illustrate reasonably valid indicators of students studying statistics who are likely to fail (Alston, Lane & Wright (2014); Elder, Jacobs & Fast (2015); Dupuis et al. (2012); Duru, Duru & Balkis (2014); Galagedera, Woodward & Degamboda (2000); Macher, Paechter et al. [2013]; Ratanaolarn (2016)). Research shows that factors such as statistical anxiety and attitude towards statistics affect the performance of students in subjects such as business statistics (Zanakis & Valenzi, 1997), research methodology (Mji & Onwuegbuzie, 2004) and introductory statistics for psychology students (Chiesi, Primi & Carmona, 2011). It has also been shown that procrastination and statistical anxiety are related (Onwuegbuzie, 2004). Studies have further revealed that statistical anxiety is influenced by previous mathematical experience and skills (Baloǧlu, 2003). Various motivational regulation strategies can be indirectly linked to high academic performance via academic procrastination (Grunschel et al., 2016) and learning effort (Schwinger & Stiensmeier-Pelster, 2012). These studies indicate that the following factors are important when determining a statistics student’s risk: statistical anxiety, attitude towards statistics, motivation, academic procrastination and mathematical foundation. Each of these concepts will be defined and its effect on performance in statistics courses, discussed. Statistical anxiety is “a performance characterized by extensive worry, intrusive thoughts, mental disorganization, tension, and physiological arousal…when exposed to statistics content, problems, instructional situations, or evaluative contexts, and is commonly claimed to debilitate performance in a wide variety of academic situations by interfering with the manipulation of statistics data and solution of statistics problems” (Zeidner, 1991: 319). Statistical anxiety is a relatively common problem among students in the social and behavioural sciences (Onwuegbuzie & Wilson, 2003; Einbinder, 2014), management sciences http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 1452022 40(2): 145-158 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 Coetzee Measuring risks associated with students of introductory statistics (Zanakis & Valenzi, 1997) and health sciences (Welch et al., 2015). It adversely affects a student’s ability to understand research articles, data analysis and the interpretation of these analyses (Onwuegbuzie, 1997). It also has an adverse effect on academic performance in statistics (Macher et al., 2013). Attitudes towards statistics can be defined as “a disposition to respond favourably or unfavourably to objects, situations or people related to statistics learning” (Chiesi & Primi, 2009: 309). Such attitudes include anxiety, cynicism, fear and even contempt (Hopkins, Glass & Hopkins, 1987). According to Ratanaolarn (2016), students’ grades in statistics are influenced positively by their attitudes towards the subject. Not only do these attitudes affect their performance, they also have an effect on the extent to which they may be willing to use statistics later in their careers (Zanakis & Valenzi, 1997, Montcalm, 1999). Academic success is influenced by various aspects of motivation; one of the important ones is self-regulation. This includes a student’s ability to direct his or her thoughts and behaviours towards the attainment of their goals (Dunn et al., 2012). Academic procrastination can be seen as the daily postponement of duties and responsibilities that relate to school (or college or university), or to delay them to the eleventh hour (McCarthy, Haycock & Skay, 1998). A significant number – between 80% and 95% – of college students apparently engage in procrastination (Wilson & Nguyen, 2012). Steel (2007) reports a correlation of –0.19 between academic performance and procrastination. The negative effect on academic performance is because passive procrastination limits not only the quantity but also the quality of a student’s work (Dunn et al., 2012). Owing to the numerical nature of statistics, students lacking a strong mathematical foundation tend to struggle with statistics. Dupuis et al. (2012) explain in their research that a strong mathematical background benefits a student later when studying statistics, since those with such a history enrolled for more difficult statistics courses and obtained higher grades. However, Galagedera et al. (2000) found no significant correlation between prior mathematical performance and statistical reasoning. 3. Research objective The primary objective of this study was to develop a scale that can be used as a detection tool to identify students at risk of failing an introductory statistics module. The dimensionality of the scale was determined, and its effectiveness was illustrated by comparing students’ risk to fail, based on the scale, with the marks they obtained. Finally, guidelines have been provided on how to use it to identify and assist at-risk students. Research questions that were addressed are: • Does the SSRQ scale meet the requirements for goodness of fit? • Is the scale reliable? • Do the items demonstrate good discrimination? • Do the constructs display discriminant validity? • Is a high mean risk (as calculated from the SSRQ) associated with a low mark in an introductory statistics course? • Is there a difference in the performance of students in statistics, based on the type of mathematics taken in matric? http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 1462022 40(2): 146-158 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 Perspectives in Education 2022: 40(2) 4. Research method The target population for this study was BA and BCom students who were enrolled for an introductory statistics module at a leading South African university. Permission was obtained from the ethics committee to conduct the research; ethical clearance number is ECONIT-2016-028. In order to develop a suitable scale and test its usability, a cross-sectional correlational study was used. First, the literature was reviewed to identify possible risks. Existing scales that measure these risks were considered. A scale was then developed – which we call the Statistics Students’ Risk Questionnaire (SSRQ) scale – using some questions from different scales to form one concise scale that measures the identified risks. Statistics lecturers were asked to comment on the scale and amendments were made. After completing a pilot study, the final questionnaire consisted of the scale’s questions as well as questions pertaining to a student’s mathematical foundation and performance in the statistics course. The questionnaire was distributed among students of an introductory statistics course, which they voluntarily completed. Construct validity was investigated using exploratory and confirmatory factor analysis. Reliability and discriminant validity were determined. To assess if the scale provides a good measure to predict which students may be at risk, the mean risk of a student was compared to the student’s performance in the course. In constructing the scale items of the Statistics Students’ Risk Questionnaire (SSRQ) used to assess the students, three resources were used: information from the literature, inputs from colleagues, and students’ opinions. Possible risks were identified: statistical anxiety, attitude towards statistics, motivation and academic procrastination. Established scales that measured these risks were identified and considered (the Statistical Anxiety Rating Scale [STARS] (Cruise, Cash & Bolton, 1985); Attitude Toward Statistics Scale [ATS] (Wise, 1985); the General Procrastination scale [GP] (Lay, 1986); the Adult Inventory of Procrastination [AIP] (Díaz-Morales et al., 2006), and Motivated Strategies for Learning Questionnaire [MSLQ] (Dunn et al., 2012). Drawing on these scales, a new scale was devised. A pilot questionnaire was constructed that contained questions that measured each of the possible risks mentioned above. Statistics lecturers were asked to evaluate the original, proposed questionnaire, and as a result some changes were made. The questionnaire was then distributed online, and 170 students voluntarily completed it. A final pool of 19 items was formed, assessed using a five-point Likert scale. The final SSRQ was distributed to 274 participants in the introductory statistics classes during the following year. The collected data were uploaded and analysed using SAS 9.4. 5. Results 5.1 Sample profile A total of 274 responses were captured for analysis. The race distribution in the sample was black (81%), white (11%), coloured (6%) and Indian (2%). Most of the students were registered for BCom (54%) and the rest for BA (46%) degrees. As many as 78% had not failed statistics modules previously, 17% had failed a statistics module once, whereas 5% had failed it twice or more times. Moreover, 54% passed mathematics and 46% passed mathematical literacy in Grade 12. http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 1472022 40(2): 147-158 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 Coetzee Measuring risks associated with students of introductory statistics 5.2 Exploratory factor analysis (EFA) Four measures were used to evaluate the appropriateness of the data set for factor analysis: sample size, sample to variable ratio, Kaiser’s Measure of Sampling Adequacy (MSA), and Bartlett’s test of sphericity. The sample size obtained was 217 (57 observations had missing values). This is acceptable because the recommended minimum sample size is 150, according to Pallant (2013). The number of variables considered for the EFA was 19. The sample to variable ratio was 11.4 cases per variable. This is greater than 5, which is the minimum acceptable ratio of cases per variable (Pallant, 2013). The overall MSA yielded a value of 0.822. According to Hair et al. (2014), a value of 0.80 or above is meritorious. A high MSA is associated with a large sample size, high average correlations and a low number of factors. MSA for the individual variables ranged from 0.70 to 0.87, which are all above the minimum requirement of 0.50 (Hair et al., 2014). In addition, Bartlett’s test of sphericity was significant (p < 0.0001). This tests if the correlation matrix shows significant correlations among the variables. The data set was considered appropriate for factor analysis (Hair et al., 2014). Three criteria were used to determine the number of factors to be extracted: latent root criterion, scree test and cumulative percentage variance explained. According to the latent root criterion, an eigenvalue represents the amount of variance associated with each factor. Four eigenvalues were greater than one. The “elbow” of the scree plot indicates that four factors should be extracted. Four factors represented 60.3% of the total variation of the 19 variables. Based on these three criteria, it was decided to extract four factors using the principal component extraction method with varimax rotation. All the constructs were one-dimensional, meaning that each construct loaded high on one of the factors but loaded significantly lower on the others. All the factor loadings were significant – above 0.40 (Hair et al., 2014). The item-total correlation of the items ranged from 0.23 to 0.53. Typically, the minimum value should not be lower than 0.30. Items 18 and 19 (out of 19) were the only variables with values less than 0.30. It was therefore decided to remove them and redo the factor analysis. After removing these two items, the total variance of the 17 items explained by the four factors was 58.1%. Preferably, the communalities should not be less than 0.50 (Hair et al., 2014). In this case, they ranged from 0.483 to 0.702, which was deemed acceptable. The results of the factor analysis are shown in Table 1 below. The four-factor SSRQ scale was further refined and validated through confirmatory factor analysis (CFA). Table 1: Results obtained from the factor analysis and item analyses Factors/ Dimensions Item No. Factor loading value Item–total correlations a Lower– upper groups difference t-value Cronbach’s 𝛼 b Eigen- value c Variance explained Statistical Anxiety 2 0.831 0.494 –8.29** 0.837 4.848 28.5% 1 0.778 0.535 –8.61** 5 0.769 0.481 –9.78** 4 0.761 0.530 –8.75** 3 0.647 0.482 –7.09** http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 1482022 40(2): 148-158 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 Perspectives in Education 2022: 40(2) Factors/ Dimensions Item No. Factor loading value Item–total correlations a Lower– upper groups difference t-value Cronbach’s 𝛼 b Eigen- value c Variance explained Procras- tination 9 0.760 0.486 –8.39** 0.789 2.125 12.5% 7 0.734 0.509 –7.59** 6 0.655 0.470 –7.58** 10 0.644 0.475 –7.32** 8 0.633 0.382 –6.01** R-At Attitude*** 13 0.764 0.382 –6.30** 0.702 1.158 6.8% 14 0.697 0.348 –7.10** 12 0.650 0.316 –4.01* 11 0.580 0.513 –8.82** R-Motivation*** 17 0.783 0.391 –5.48** 0.620 1.758 10.3% 16 0.633 0.373 –6.67* 15 0.552 0.418 –6.83** *p < 0.001 **p < 0.0001 ***Attitude and motivation were reverse coded. a. Item–total correlations are all above 0.3. b. Cronbach’s 𝛼 values are all above 0.6. c. Eigenvalues are all above 1. 5.3 Confirmatory factor analysis The indicator items measured were assigned to latent constructs. CFA was run for all the constructs through structural equation modelling (SEM), using Proc Calis of SAS 9.4. The SSRQ scale with four factors collectively comprised 17 items. Various goodness-of-fit indices were used to assess the SSRQ scale. SEM allows for testing of the interrelationships among a set of related variables through multiple regression procedures. The overall fit of the model is indicated by the chi-square test (χ2). For the SSRQ scale χ2 = 256.82, df = 113, p < 0.0001 was obtained. However, this test is viewed as an overly strict indicator of model fit, given its power to detect even inconsequential deviations from the proposed model (Hair et al., 2014). The normed chi-square ratio χ2/df should ideally be less than 3 (Hair et al., 2014). The SSRQ scale met the required condition, since χ2/df = 2.273. A number of model fit indices were used to assess model fit: goodness-of-fit index (GFI), adjusted goodness-of-fit index (AGFI) and comparative fit index (CFI). A perfect fit is obtained when indices are close to one (1), whereas those close to zero (0) represent no fit (Hair et al., 2014). The results of the fit indices were 0.888 for GFI, 0.849 for AGFI, and 0.880 for CFI. Concerning the root mean square error of approximation (RMSEA), there is a good model fit if RMSEA is less than or equal to 0.05 and an adequate fit if it is less than or equal to 0.08 (Blunch, 2016). The RMSEA result for this model was 0.073. Overall, the scale portrayed a very good fit for the data set as indicated by the chi-square ratio test, several goodness-of-fit indices, and RMSEA results. Thus, the 17-item scale SSRQ with four dimensions is valid and robust. http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 1492022 40(2): 149-158 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 Coetzee Measuring risks associated with students of introductory statistics 5.4 Reliability The overall SSRQ scale displayed excellent reliability (Cronbach’s α = 0.84). The Cronbach’s 𝛼 reliability of three of the four factors extracted ranged from 0.702 to 0.837. The Cronbach’s 𝛼 reliability of motivation was 0.62, which is somewhat low, but still acceptable (Hair et al., 2014). Cronbach’s 𝛼 reliability of the four factors extracted were all above 0.6, which portray fairly good reliability. Composite reliability (CR) for statistical anxiety was 0.87, for procrastination, 0.82, for attitude, 0.77 and for motivation, 0.70. This suggests good internal consistency of the scale. Thus, overall, the scale was found to be reliable. 5.5 Item analyses The results of the item analyses are presented in Table 1. The item–total point correlation of the items in the SSRQ yielded a minimum value of 0.316 and a maximum value of 0.535. Since all the values are higher than 0.30, the items’ discrimination is good. The total scale point was calculated, and the 27th and 73rd percentiles were calculated. Values below the 27th percentile were allocated to the lower group, whereas values above the 73rd percentile were allocated to the upper group. T-tests were used to determine if the items’ means differed significantly for the two groups. All the t-values were found to be significant (p < 0.001), indicating that the distinctiveness levels for all items are high. Each item, therefore, makes a consistent classification. 5.6 Discriminant validity Discriminant validity refers to the extent to which the constructs differ from one another (Hamid, Sami & Sidek, 2017). One measure to evaluate discriminant validity is the Fornell and Larcker criterion. Here, the square root of the average variance extracted (AVE) is compared with the correlation of latent constructs (Hamid et al., 2017). The reasoning behind this criterion is that a latent construct should explain the variance of its own indicator better than the variance of other latent constructs. Therefore, the square root of each construct’s AVE should be greater than each of the correlations with other latent constructs (Hamid et al., 2017). As can be seen in Table 2 below, the smallest square root value of AVE is 0.620, which is greater than the largest squared correlation of 0.330. According to the Fornell and Larcker criterion, the scale shows discriminant validity. Table 2: Squared correlations and the square root of AVE — average variance extracted (on the diagonal) SSRQ dimensions Statistical anxiety Procrastination Attitude Motivation Statistical anxiety 0.719 Procrastination 0.164 0.646 Attitude 0.138 0.082 0.622 Motivation 0.074 0.330 0.301 0.620 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 1502022 40(2): 150-158 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 Perspectives in Education 2022: 40(2) 5.7 Scoring the scale For each individual whose risk of performing badly in introductory statistics is to be measured, 1, 2, 3, 4 or 5 points were given to the options “Strongly disagree”, “Disagree”, “Neutral”, “Agree”, and “Strongly agree”, respectively. The points were added together and divided by the number of items, to provide a measure of the risk of performing badly. The expected point range is 1.0 (lowest) and 5.0 (highest), where a value of close to 1.0 indicates that the student is not at high risk, whereas a value close to 5.0 indicates that he or she is at high risk. 5.8 Identify students at risk The mean risk was calculated for each participant in the survey as explained above and compared to the student’s participation mark (made up of marks for tests and assignments). As can be seen from the statistics provided in Table 3, low participation marks are associated with high mean risk, and high participation marks with low mean risk. The correlation between the mean risk and the participation mark is r = –0.446 (p < 0.0001), indicating a moderately negative relationship. Correlations between the participation mark and the individual factors are r = –0.322 (p < 0.0001) for statistical anxiety, r = –0.232 (p = 0.0002) for academic procrastination, r = 0.353 (p < 0.0001) for motivation and r = 0.328 (p < 0.0001) for attitude towards statistics. The negative association found between performance and statistical anxiety is stronger than the correlation (r = –0.035) reported by Paechter et al. (2013), who used the STARS instrument. The results of the SSRQ also compare favourably with the correlation of –0.19 that Steel (2007) reports between academic performance and procrastination. Chiesi & Primi (2010) also found a positive effect between performance and attitude. Students awarded a participation mark of 59 or less can be considered still at risk of failing the exam, therefore a cut-off value of 2.80 may be used. Students whose mean risk value is higher than 2.80 may be at risk of failing the module, whereas those with a mean risk of less than 2.80 have a low risk of failing. Table 3: Risk statistics for different intervals of the participation mark: mean, standard deviation and sample size Participation mark Mean risk Standard deviation of risk Sample size 0–39 2.98 0.75 12 40–49 2.89 0.58 29 50–59 2.93 0.63 42 60–69 2.70 0.50 58 70–79 2.44 0.58 54 80+ 1.99 0.57 35 The analysis of variance indicates that the mean risk differs significantly for different participation mark groupings (F = 13.92, p < 0.0001). This data set does not indicate a linear relationship. However, the fact that the number of students in the first two classes (0–39 and 40–49) is small (n = 12 and 29, respectively) may be the reason why these two classes behave differently from the expected. No model can perfectly distinguish performers from non- performers because other factors often play a role that were not determined at the time the study was conducted, for example illness. Another way of identifying students at risk is by looking at how many of the four factors have a mean value greater than 3. As Figure 1 indicates, the more factors that have mean http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 1512022 40(2): 151-158 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 Coetzee Measuring risks associated with students of introductory statistics values greater than 3, the higher the mean risk of a student failing. A student with no risk factors with values greater than 3 will typically have a low risk of failing. When a student has two or more factors that have been flagged, the risk becomes high, and action should be taken to support that person. Figure 1: Risk factors (having mean values greater than 3) versus mean risk. The more factors that have been flagged as representing a risk, the higher the mean risk is. Another factor that may be added to the four factors of the SSRQ is the student’s mathematical background. In South African state schools, learners can matriculate either with mathematics or with mathematical literacy. This study indicates that students who matriculated with mathematical literacy are at a significantly greater risk of performing poorly in statistics than those who matriculated with mathematics. Of the students who scored less than 50% for their participation marks, 69% had matriculated with mathematical literacy. On the other hand, 76% of the students who obtained 70% or more had taken mathematics ( = 33.16, df = 3, p < 0.0001). Figure 2 shows the relationship between mathematical type and participation mark. http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 1522022 40(2): 152-158 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 Perspectives in Education 2022: 40(2) 12 Figure 2: The effect of the mathematics exam taken for matriculation on participation mark. Having matriculated with mathematical literacy is more associated with low participation marks whereas having mathematics as a subject in Grade 12 is more associated with high participation marks. 6. Discussion 6.1 Development and performance of scale The SSRQ scale was developed to measure the potential risk that a university student will fail an introductory statistics module. The aim of the tool was to have a measure that quantifies a student’s risk of failing a statistics module early in the semester, so that these at- risk students can be identified and special care provided for them. The results have shown that the SSRQ scale consists of four dimensions that are valid and robust. The overall scale shows excellent reliability with good internal consistency, while each of the factors individually has satisfactory reliability. The scales’ items show good discriminant validity. The SSRQ consists of 17 items, each measured on a five-point Likert scale. The PCA revealed four underlying factors, namely Statistical Anxiety, Attitude towards Statistics, Motivation, and Academic Procrastination. These factors accounted for 58.1% of the total variance. High levels of statistical anxiety and academic procrastination, together with low motivational levels and a negative attitude towards statistics are indicators of an at-risk student. The Cronbach’s alpha reliability of the whole scale was 0.84, which compares well with 0.88 for STARS and 0.82 for SATS that were administered to South African students (Mji, 2009). The scale compares well with a scale developed by Ratanaolarn 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0-49 50-59 60-69 70+ P er ce nt ag e of s tu de nt s Participation mark Math type Math type Figure 2: The effect of the mathematics exam taken for matriculation on participation mark. Having matriculated with mathematical literacy is more associated with low participation marks whereas having mathematics as a subject in Grade 12 is more associated with high participation marks. 6. Discussion 6.1 Development and performance of scale The SSRQ scale was developed to measure the potential risk that a university student will fail an introductory statistics module. The aim of the tool was to have a measure that quantifies a student’s risk of failing a statistics module early in the semester, so that these at-risk students can be identified and special care provided for them. The results have shown that the SSRQ scale consists of four dimensions that are valid and robust. The overall scale shows excellent reliability with good internal consistency, while each of the factors individually has satisfactory reliability. The scales’ items show good discriminant validity. The SSRQ consists of 17 items, each measured on a five-point Likert scale. The PCA revealed four underlying factors, namely Statistical Anxiety, Attitude towards Statistics, Motivation, and Academic Procrastination. These factors accounted for 58.1% of the total variance. High levels of statistical anxiety and academic procrastination, together with low motivational levels and a negative attitude towards statistics are indicators of an at-risk student. The Cronbach’s alpha reliability of the whole scale was 0.84, which compares well with 0.88 for STARS and 0.82 for SATS that were administered to South African students (Mji, 2009). The scale compares well with a scale developed by Ratanaolarn (2016), which combined STARS, SATS and the Course Experience Questionnaire. The χ2/df of the CFA of SSRQ was found to be 2.27 (compared to 1.07 of Ratanaolarn’s instrument) and the RMSEA was 0.07 (versus 0.02). Furthermore, the GFI, AGFI and CFI were 0.89 (versus 0.93), 0.85 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 1532022 40(2): 153-158 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 Coetzee Measuring risks associated with students of introductory statistics (versus 0.91) and 0.88 (versus 0.99), respectively. The CR ranged from 0.697 to 0.871 for the four factors, indicating good reliability. The Fornell and Larcker criterion had a minimum square root value of 0.620, which is greater than the maximum squared correlation of 0.330, indicating discriminant validity. The Lower–Upper Groups Difference t-values are all significant (p < 0.001), indicating that each item makes a consistent classification. When comparing the SSRQ to the SATS and the STARS, which are widely used to determine attitudes towards statistics and statistical anxiety, it should be noted that the SATS consists of 36 response items and 51 for the STARS. Regarding procrastination, the GP and the AIP scales consist of 20 and 15 items, respectively. The MSLQ scale, which measures motivation, consists of 81 items. The final SSRQ in this study featured only 17 items, measuring not only attitude towards statistics and statistical anxiety, but also academic procrastination and motivation. The SSRQ can obviously not measure each of the factors in the same depth as the established other scales mentioned above. However, its advantage is that it is relatively short and can easily be administered during a class to all the students to determine which, if any, of the risk factors may potentially cause any of them to struggle. Since the SSRQ is a short questionnaire, the chances of careless responding due to questionnaire fatigue may be minimised, as proposed by Gibson & Bowling (2020). The mean risk for each student was calculated based on the SSRQ scale and compared to the student’s marks. High mean risks were associated with low marks and vice versa (r = –0.446), indicating the ability of the scale correctly to detect at-risk students. 6.2 Implementation of scale This study therefore puts a tool in the hands of a statistics lecturer to determine which students are at risk. To implement the scale in practice, the lecturer can assess the overall risk by calculating the mean risk over all the factors. If this mean risk is greater than 2.80, a student is considered to be at risk. However, the lecturer can also look at which specific factors are flagged for a particular individual. If only one factor is indicated – that is, the mean value of a particular factor is greater than 3 – the risk is relatively low. As the number of flagged factors increases, the risk of the particular student failing, increases. Another factor that can be added to the risk assessment instrument is whether a student matriculated with mathematical literacy, since it was found that undergraduates with this particular qualification in matric have a significantly lower participation mark on average than their counterparts who matriculated with mathematics. 6.3 Strategy to assist at-risk students By considering which factors contribute to the risk, a customised plan of action can be adopted. If, for instance, it is found that a student has a problem with anxiety, this person may complete the STARS in a follow-up session and thereafter be referred to campus psychological support services. Similarly, the SSRQ may be followed by administering the SATS in the case of an attitude problem, the AIP or GP when faced with a procrastination issue, and the MSLQ to address a lack of motivation. The lecturer may address attitude and motivation by showing students how the statistics module links to their major and where mastering this skill will be used in the workplace. Students who tend to procrastinate may be required to undergo time management training. Those with mathematical literacy may need to take a mathematical bridging course before starting a statistics programme. http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 1542022 40(2): 154-158 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 Perspectives in Education 2022: 40(2) 6.4 Strengths and limitations This study is limited by being conducted at only one South African university. It is desirable to repeat it elsewhere – in South Africa as well as abroad – to make it more generalisable. Although the sample size was relatively small, it did provide enough power for the EFA and the CFA. Future studies are needed to investigate test–retest reliability and criterion validity. The strengths of this research are the reliability and discriminant validity of the overall scale and subscales, high PCA loadings and the clarity of the four-factor model derived from the PCA, which is supported by the CFA, as well as significant correlations between each of the risk factors and academic performance. 7. Conclusion The SSRQ scale was shown to be effective to detect at-risk students in an introductory statistics module. Lecturers can implement the scale to address students’ need to receive the right support, for example assistance to address their statistical anxiety, their negative attitude towards statistics, their lack of motivation to study statistics and their tendency to procrastinate academically once they are revealed to be “at risk”. These are all “soft” skills that can be addressed to assist students in their journey to succeed in acquiring the much-needed statistical skills. Where students have an inadequate mathematical background, this may be addressed by a bridging course in mathematics. 8. Disclosure statement No potential conflict of interest was reported by the author. References Alston, G.L., Lane, D. & Wright, N.J.D. 2014. Research: The methodology for the early identification of students at risk for failure in a professional degree program. Currents in Pharmacy Teaching and Learning, 6: 798-806. https://doi.org/10.1016/j.cptl.2014.07.006 Baloǧlu, M. 2003. Individual differences in statistics anxiety among college students. Personality and Individual Differences, 34: 855-865. https://doi.org/10.1016/S0191-8869(02)00076-4 Blunch, N.J. 2016. Introduction to structural equation modeling using IBM SPSS statistics and EQS. Los Angeles: SAGE. https://doi.org/10.4135/9781473922273 Chiesi, F. & Primi, C. 2009. Assessing statistics attitudes among college students: Psychometric properties of the Italian version of the Survey of Attitudes toward Statistics (SATS). Learning and Individual Differences, 19: 309-313. https://doi.org/10.1016/j.lindif.2008.10.008 Chiesi, F. & Primi, C. 2010. Cognitive and non-cognitive factors related to students’ statistics achievement. Statistics Education Research Journal, 9: 6-26. https://doi.org/10.52041/serj. v9i1.385 Chiesi, F., Primi, C. & Carmona, J. 2011. Measuring statistics anxiety: Cross-country validity of the Statistical Anxiety Scale (SAS). Journal of Psychoeducational Assessment, 29: 559-569. https://doi.org/10.1177/0734282911404985 Cruise, R.J., Cash, R.W. & Bolton, D.L. Development and validation of an instrument to measure statistical anxiety. Paper presented at the annual meeting of the Statistical Education Section; 1985 Aug in Chicago, IL. http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 https://doi.org/10.1016/j.cptl.2014.07.006 https://doi.org/10.1016/S0191-8869(02)00076-4 https://doi.org/10.4135/9781473922273 https://doi.org/10.1016/j.lindif.2008.10.008 https://doi.org/10.52041/serj.v9i1.385 https://doi.org/10.52041/serj.v9i1.385 https://doi.org/10.1177/0734282911404985 1552022 40(2): 155-158 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 Coetzee Measuring risks associated with students of introductory statistics Díaz-Morales, J. F., Ferrari, J. R., Díaz, K. & Argumedo, D. 2006. Factorial structure of three procrastination scales with a Spanish adult population. European Journal of Psychological Assessment, 22: 132-137. https://doi.org/10.1027/1015-5759.22.2.132 Dunn, K.E., Lo, W.-J., Mulvenon, S.W. & Sutcliffe, R. 2012. Revisiting the motivated strategies for learning questionnaire: A theoretical and statistical re-evaluation of the metacognitive self-regulation and effort regulation subscales. Educational and Psychological Measurement, 72: 312-331. https://doi.org/10.1177/0013164411413461 Dupuis, D.N., Medhanie, A., Harwell, M., Lebeau, B., Monson, D. & Post, T.R. 2012. A multi-institutional study of the relationship between high school mathematics achievement and performance in introductory college statistics. Statistics Education Research Journal, 11: 4-20. https://doi.org/10.52041/serj.v11i1.337 Duru, E., Duru, S. & Balkis, M. 2014. Analysis of relationships among burnout, academic achievement, and self-regulation. Educational Sciences: Theory and Practice, 14: 1274-1284. https://doi.org/10.12738/estp.2014.4.2050 Einbinder, S.D. 2014. Reducing research anxiety among MSW students. Journal of Teaching in Social Work, 34: 2-16. https://doi.org/10.1080/08841233.2013.863263 Elder, B.L., Jacobs, P. & Fast, Y.J. 2015. Identification and support of at-risk students using a case management model. Journal of Professional Nursing, 31: 247-253. https://doi. org/10.1016/j.profnurs.2014.10.003 Galagedera, D., Woodward, G. & Degamboda, S. 2000. An investigation of how perceptions of mathematics ability can affect elementary statistics performance. International Journal of Mathematical Education in Science & Technology, 31: 679-689. https://doi. org/10.1080/002073900434369 Gibson, A.M. & Bowling, N.A. 2020. The effects of questionnaire length and behavioral consequences on careless responding. European Journal of Psychological Assessment, 36: 410-420. https://doi.org/10.1027/1015-5759/a000526 Grunschel, C., Schwinger, M., Steinmayr, R. & Fries, S. 2016. Effects of using motivational regulation strategies on students’ academic procrastination, academic performance, and well-being. Learning and Individual Differences, 49: 162-170. https://doi.org/10.1016/j. lindif.2016.06.008 Gultice, A., Witham, A. & Kallmeyer, R. 2015. Are your students ready for anatomy and physiology? Developing tools to identify students at risk for failure. Advances in Physiology Education, 39: 108-115. https://doi.org/10.1152/advan.00112.2014 Hair, J.F., Black, W.C., Babin, B.J. & Anderson, R.E. 2014. Multivariate data analysis. UK: Pearson Education Limited. Hamid, M.R.A., Sami, W. & Sidek, M.H.M. 2017. Discriminant validity assessment: Use of Fornell & Larcker criterion versus HTMT criterion. Journal of Physics: Conference Series, 890: 012163. https://doi.org/10.1088/1742-6596/890/1/012163 Hopkins, K.D., Glass, G.V. & Hopkins, B.R. 1987. Basic statistics for the behavioral sciences, 2nd edition. Englewood Cliffs, NJ, US: Prentice-Hall, Inc. Lay, C.H. 1986. At last, my research article on procrastination. Journal of Research in Personality, 20: 474-495. https://doi.org/10.1016/0092-6566(86)90127-3 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 https://doi.org/10.1027/1015-5759.22.2.132 https://doi.org/10.1177/0013164411413461 https://doi.org/10.52041/serj.v11i1.337 https://doi.org/10.12738/estp.2014.4.2050 https://doi.org/10.1080/08841233.2013.863263 https://doi.org/10.1016/j.profnurs.2014.10.003 https://doi.org/10.1016/j.profnurs.2014.10.003 https://doi.org/10.1080/002073900434369 https://doi.org/10.1080/002073900434369 https://doi.org/10.1027/1015-5759/a000526 https://doi.org/10.1016/j.lindif.2016.06.008 https://doi.org/10.1016/j.lindif.2016.06.008 https://doi.org/10.1152/advan.00112.2014 https://doi.org/10.1088/1742-6596/890/1/012163 https://doi.org/10.1016/0092-6566(86)90127-3 1562022 40(2): 156-158 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 Perspectives in Education 2022: 40(2) León-Mantero, C., Casas-Rosal, J.C., Maz-Machado, A. & Rico, M.E.V. 2020. Analysis of attitudinal components towards statistics among students from different academic degrees. PLoS ONE, 15: 1-13. https://doi.org/10.1371/journal.pone.0227213 Macher, D., Paechter, M., Papousek, I., Ruggeri, K., Freudenthaler, H.H. & Arendasy, M. 2013. Statistics anxiety, state anxiety during an examination, and academic achievement. British Journal of Educational Psychology, 83: 535-549. https://doi.org/10.1111/j.2044-8279.2012.02081.x Mccarthy, P., Haycock, L.A. & Skay, C.L. 1998. Procrastination in college students: The role of self-efficacy and anxiety. Journal of Counseling and Development, 76: 317-324. https://doi. org/10.1002/j.1556-6676.1998.tb02548.x Mji, A. & Onwuegbuzie, A.J. 2004. Evidence of score reliability and validity of the Statistical Anxiety Rating Scale among technikon students in South Africa. Measurement and Evaluation in Counseling and Development: 238. https://doi.org/10.1080/07481756.2004.11909745 Montcalm, D.M. 1999. Applying Bandura’s theory of self-efficacy to the teaching of research. Journal of Teaching in Social Work, 19: 93-107. https://doi.org/10.1300/J067v19n01_08 Onwuegbuzie, A.J. 1997. Writing a research proposal: The role of library anxiety, statistics anxiety, and composition anxiety. Library and Information Science Research, 19: 5-33. https:// doi.org/10.1016/S0740-8188(97)90003-7 Onwuegbuzie, A.J. 2004. Academic procrastination and statistics anxiety. Assessment & Evaluation in Higher Education, 29: 3-19. https://doi.org/10.1080/0260293042000160384 Onwuegbuzie, A.J. & Wilson, V.A. 2003. Statistics anxiety: nature, etiology, antecedents, effects, and treatments – a comprehensive review of the literature. Teaching in Higher Education, 8: 195. https://doi.org/10.1080/1356251032000052447 Ramirez, C., Schau, C. & Emmioglu, E. 2012. The importance of attitudes in statistics education. Statistics Education Research Journal, 11(2): 57-71. https://doi.org/10.52041/serj. v11i2.329 Ratanaolarn, T. 2016. The development of a structural equation model of graduate students’ statistics achievement. International Journal of Behavioral Science, 11: 153-168. Ridgway, J., Nicholson, J. & Mccusker, S. 2007. Teaching statistics – despite its applications. Teaching Statistics, 29: 44-48. https://doi.org/10.1111/j.1467-9639.2007.00276.x Schwinger, M. & Stiensmeier-Pelster, J. 2012. Effects of motivational regulation on effort and achievement: A mediation model. International Journal of Educational Research, 56: 35-47. https://doi.org/10.1016/j.ijer.2012.07.005 Steel, P. 2007. The nature of procrastination: A meta-analytic and theoretical review of quintessential self-regulatory failure. Psychological Bulletin, 133: 65-94. https://doi. org/10.1037/0033-2909.133.1.65 Welch, P.S., Jacks, M.E., Smiley, L.A., Walden, C.E., Clark, W.D. & Nguyen, C.A. 2015. A study of statistics anxiety levels of graduate dental hygiene students. Journal of Dental Hygiene, 89: 46-54. Wise, S.L. 1985. The development and validation of a scale measuring attitudes toward statistics. Educational and Psychological Measurement, 45: 401-405. https://doi. org/10.1177/001316448504500226 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 https://doi.org/10.1371/journal.pone.0227213 https://doi.org/10.1111/j.2044-8279.2012.02081.x https://doi.org/10.1002/j.1556-6676.1998.tb02548.x https://doi.org/10.1002/j.1556-6676.1998.tb02548.x https://doi.org/10.1080/07481756.2004.11909745 https://doi.org/10.1300/J067v19n01_08 https://doi.org/10.1016/S0740-8188(97)90003-7 https://doi.org/10.1016/S0740-8188(97)90003-7 https://doi.org/10.1080/0260293042000160384 https://doi.org/10.1080/1356251032000052447 https://doi.org/10.52041/serj.v11i2.329 https://doi.org/10.52041/serj.v11i2.329 https://doi.org/10.1111/j.1467-9639.2007.00276.x https://doi.org/10.1016/j.ijer.2012.07.005 https://doi.org/10.1037/0033-2909.133.1.65 https://doi.org/10.1037/0033-2909.133.1.65 https://doi.org/10.1177/001316448504500226 https://doi.org/10.1177/001316448504500226 1572022 40(2): 157-158 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 Coetzee Measuring risks associated with students of introductory statistics Wilson, B.A. & Nguyen, T.D. 2012. Belonging to tomorrow: an overview of procrastination. International Journal of Psychological Studies, 4(1): 211-217. https://doi.org/10.5539/ijps. v4n1p211 Zanakis, S.H. & Valenzi, E.R. 1997. Student anxiety and attitudes in business statistics. Journal of Education for Business, 73: 10. https://doi.org/10.1080/08832329709601608 Zeidner, M. 1991. Statistics and mathematics anxiety in social science students: Some interesting parallels. British Journal of Educational Psychology, 61: 319-328. https://doi. org/10.1111/j.2044-8279.1991.tb00989.x Appendix: The Statistics Students’ Risk Questionnaire (SSRQ) Please note: Items 11 to 17 have been reverse coded (R) for all the analyses Item Question 1 2 3 4 5 1 I experience feelings of anxiety when I have to attend a statistics class. S tro ng ly di sa gr ee D is ag re e N eu tra l A gr ee S tro ng ly ag re e 2 I experience feelings of anxiety when I have to solve a statistical problem. S tro ng ly di sa gr ee D is ag re e N eu tra l A gr ee S tro ng ly ag re e 3 I experience feelings of anxiety when I need to ask the statistics lecturer a question. S tro ng ly di sa gr ee D is ag re e N eu tra l A gr ee S tro ng ly ag re e 4 I experience feelings of anxiety when I have to study for a statistics test. S tro ng ly di sa gr ee D is ag re e N eu tra l A gr ee S tro ng ly ag re e 5 I experience feelings of anxiety while I am writing a statistics test. S tro ng ly di sa gr ee D is ag re e N eu tra l A gr ee S tro ng ly ag re e 6 I generally delay before starting work I have to do for this module. S tro ng ly di sa gr ee D is ag re e N eu tra l A gr ee S tro ng ly ag re e 7 In preparing for deadlines for this module, I often waste time by doing other things. S tro ng ly di sa gr ee D is ag re e N eu tra l A gr ee S tro ng ly ag re e 8 I often start my statistics homework at the last minute and find it difficult to complete it on time. S tro ng ly di sa gr ee D is ag re e N eu tra l A gr ee S tro ng ly ag re e 9 I tend to procrastinate (wait until last minute) more when studying for statistics than studying for other modules. S tro ng ly di sa gr ee D is ag re e N eu tra l A gr ee S tro ng ly ag re e http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 https://doi.org/10.5539/ijps.v4n1p211 https://doi.org/10.5539/ijps.v4n1p211 https://doi.org/10.1080/08832329709601608 https://doi.org/10.1111/j.2044-8279.1991.tb00989.x https://doi.org/10.1111/j.2044-8279.1991.tb00989.x 1582022 40(2): 158-158 http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 Perspectives in Education 2022: 40(2) Item Question 1 2 3 4 5 10 Because I tend to wait until it is almost too late before starting to study statistics, I don’t study efficiently for tests and/or exams. S tro ng ly di sa gr ee D is ag re e N eu tra l A gr ee S tro ng ly ag re e 11 (R) My attitude towards the content of the statistics course I am doing is V er y ne ga tiv e N eg at iv e N eu tra l P os iti ve Ve ry po si tiv e 12 (R) My attitude towards my statistics lecturer is V er y ne ga tiv e N eg at iv e N eu tra l P os iti ve Ve ry po si tiv e 13 (R) I think statistics will be useful for my job S tro ng ly di sa gr ee D is ag re e N eu tra l A gr ee S tro ng ly ag re e 14 (R) I can see how statistics relate to my field of study S tro ng ly di sa gr ee D is ag re e N eu tra l A gr ee S tro ng ly ag re e 15 (R) Even when statistics course materials are dull and uninteresting, I manage to keep working until I finish. S tro ng ly di sa gr ee D is ag re e N eu tra l A gr ee S tro ng ly ag re e 16 (R) When I become confused about something I’m reading for this class, I go back and try to figure it out. S tro ng ly di sa gr ee D is ag re e N eu tra l A gr ee S tro ng ly ag re e 17 (R) The most satisfying thing for me in this course is trying to understand the content as thoroughly as possible. S tro ng ly di sa gr ee D is ag re e N eu tra l A gr ee S tro ng ly ag re e D ro pp ed I work hard to do well in statistics even when I don’t like what we are doing. S tro ng ly di sa gr ee D is ag re e N eu tra l A gr ee S tro ng ly ag re e D ro pp ed I ask myself questions to make sure I understand the material I have been studying in this class. S tro ng ly di sa gr ee D is ag re e N eu tra l A gr ee S tro ng ly ag re e http://dx.doi.org/10.18820/2519593X/pie.v40.i2.11 IDX77 IDX78 IDX79 IDX132 IDX269 IDX684 IDX759