197 Research Article 2021 39(3): 197-213 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 Published by the UFS http://journals.ufs.ac.za/index.php/pie © Creative Commons With Attribution (CC-BY) DETERMINING THE NEEDS OF INTRODUCTORY STATISTICS UNIVERSITY STUDENTS: A QUALITATIVE SURVEY STUDY ABSTRACT Since we are entering the fourth industrial revolution, it becomes increasingly important that students, irrespective of their field of study, are data literate. However, many students view their mandatory statistics modules with contempt. This qualitative study analysed students’ comments to determine what their needs are to improve these modules to increase the students’ enjoyment and improve their performance in statistics. Monte Carlo simulations were used to scientifically justify that data saturation was attained by the survey study. This is an objective reproducible method to determine if a sample size is large enough to ensure that important themes are not missed. It is clear from the students’ responses that they do not enjoy being taught in the same way students were taught statistics years ago. Statistics education needs to change to keep up with changing times and the needs of the new generation. The comments of the students touched on similar topics that are being addressed internationally regarding the instruction and assessment of statistics students; therefore, it will be sensible to implement these recommendations in future. Keywords: Statistics education research; enjoyment; performance; qualitative survey; data saturation; Monte Carlo simulations. 1. INTRODUCTION This paper describes an exploratory qualitative study in which students studying at a South African university were asked to complete a survey that contained open- ended questions. The purpose was to determine what students’ experiences were of their introductory statistics modules and what suggestions they had to improve these modules to make them more enjoyable, as well as to improve performance in these courses. Their responses were compared to the recommendations of the Guidelines for Assessment and Instruction In Statistics Education (GAISE) college report (Wood et al., 2018). Educational institutions and lecturers can use the resulting information to improve statistics modules so that students’ attitudes towards statistics may become more enthusiastic, and their ability to use statistics in everyday life, as well as in their careers, may be improved. AUTHOR: Ms Wilma Coetzee1 AFFILIATION: 1North-West University, South Africa DOI: http://dx.doi. org/10.18820/2519593X/pie.v39. i3.15 e-ISSN 2519-593X Perspectives in Education 2021 39(3): 197-213 PUBLISHED: 16 September 2021 RECEIVED: 20 November 2020 ACCEPTED: 10 March 2021 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 http://www.statssa.gov.za/?p=11341 http://documents.worldbank.org/curated/en/530481521735906534/Overcoming-Poverty-and-Inequality-in-South-Africa-An-Assessment-of-Drivers-Constraints-and-Opportunities http://documents.worldbank.org/curated/en/530481521735906534/Overcoming-Poverty-and-Inequality-in-South-Africa-An-Assessment-of-Drivers-Constraints-and-Opportunities http://documents.worldbank.org/curated/en/530481521735906534/Overcoming-Poverty-and-Inequality-in-South-Africa-An-Assessment-of-Drivers-Constraints-and-Opportunities http://orcid.org/0000-0002-7418-4581 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 1982021 39(3): 198-213 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 Perspectives in Education 2021: 39(3) The GAISE report was compiled in the USA and provides useful guidelines to enhance the teaching of statistics. Some of the guidelines it presented include the following: Statistical thinking, conceptual understanding and the statistical analysis process should be taught rather than tools and recipes provided in isolation; students have to be exposed to real data that have a context and a purpose; active learning should be fostered; technology should be used so that the focus is on analysing the data and interpreting the results rather than performing computational mechanics; assessments should be used to enhance learning – lecturers should ensure that adequate formative assessments are completed (Wood et al., 2018). These guidelines would probably be useful for other countries such as South Africa. However, before simply implementing these recommendations, it is wise to tap into the perspectives and views of the students concerned, lest we fall into the trap of thinking that we as educators know better than them what and how they need to learn (Cook-Sather, 2002). It is important to listen to the students so that changes that are made to statistical instruction in future are more likely to meet the needs of young people (Warren & Marciano, 2018). To ensure that the student sample used was adequate to identify all the important perceptions of the students regarding the two questions (enjoyment and performance), Monte Carlo simulations were applied to determine if data saturation was reached. This paper explores the voice of South African university students of statistics using qualitative techniques to ascertain their needs and ideas on a statistics course. Data saturation is determined using a technique that is not well known and has not yet been implemented in the field of statistics education. Recommendations are made on how introductory statistics education at tertiary level can be improved to address the needs of the students and thereby better prepare them for their careers. 1.1 Research approach Since the objective of this survey was to explore what would make students enjoy statistics more, and what would make them perform better, an interpretive research philosophy was adopted. The primary goal of interpretive qualitative research is to explore the participants’ experiences and to understand and interpret these experiences (Tavakol & Sandars, 2014). According to Hanson, Balmer and Giardino (2011), suitable topics for this research paradigm include programme evaluation, needs assessment, opinions, attitudes and challenges. Theories should not be developed from the researcher’s hypotheses but rather from participant perceptions (Kennedy & Lingard, 2006). As stated by Tavakol and Sandars (2014), researchers should refrain from making their own assumptions and hypotheses, in order to ensure objectivity. The preferred method advocated to collect data for qualitative research is through interviews, focus groups and/or audio-visual data, whereas qualitative surveys are rarely mentioned in textbooks on research methodology (Jansen, 2010). Such surveys with open- ended questions are inexpensive means to discover the essentials of a phenomenon (Jansen, 2010). In open quality surveys, appropriate topics are identified through the analysis of raw data that are obtained from open-ended questions. In most qualitative research projects, the focus is on the iteration of data collection and analysis, which forms the basis of grounded theory (Ramani & Mann, 2016). However, Jansen (2010) argues that many qualitative projects are built on a ”single one-shot, one-method sample” that consists of only one cycle of data collection and analysis. While this is not necessarily the best way of conducting qualitative research, one should keep in mind what the primary purpose of a qualitative study is. Usually, http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 1992021 39(3): 199-213 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 Coetzee Determining the needs of introductory statistics university students it is to explore the participants’ views as they have expressed them in their own words. It is important that the diversity of the phenomenon studied is represented by the sample. In this study, it was decided to use a qualitative survey design because it is a relatively fast and easy way to explore the views of many students. Another benefit is that respondents can freely express their views given the anonymous nature of a survey. 1.2 Listening to the voice of the students In order to determine what the young people’s opinions and recommendations are regarding the teaching of statistics, they were asked to respond to two open-ended statements, namely, “I would enjoy statistics more if…” and “I would perform better in statistics if…” 1.2.1 Enjoyment Enjoyment can be defined as “a pleasant activating emotion” (Putwain et al., 2018). In contrast, boredom is having trouble remaining alert and continuing work (Schukajlow & Rakoczy, 2016). Enjoyment of a course has a positive effect on a student’s performance (Mega, Ronconi & De Beni, 2014; Schukajlow & Rakoczy, 2016). Rowe, Fitness and Wood (2015) found that positive emotions in learning are associated with enhanced cognitive functioning, increased motivation as well as amplified self-efficacy, coping and resilience. It is thus important that students enjoy courses to enable them to perform well. 1.2.2 Performance In addition to enjoyment, many other factors influence students’ performance. Numerous quantitative studies have been conducted to ascertain the indicators of students who are likely to fail. Factors that have a positive effect on their performance in statistics include the following: mathematical ability (Jaafar et al., 2012), motivation (Morris & Fritz, 2015; Dunn et al., 2012), attitude towards statistics (Ratanaolarn, 2016) and an engaging classroom environment (Nguyen, Charity & Robson, 2016). 1.3 Data saturation Large samples are not needed in qualitative research since each participant may generate several concepts and perspectives (Starks & Trinidad, 2007). However, it is important that a qualitative researcher does not assume that the sample size used is adequate, but evaluates the sample size to determine if data saturation has been reached (Tran et al., 2016). According to Guest, Bunce and Johnson (2006), data saturation is reached when additional responses acquired from participants produce little or no new information. Surveys using open- ended questions require larger sample sizes than interview or focus group studies, because the data collection and analysis is sequential rather than iterative (Tran et al., 2016). Views differ on the identification of data saturation and when sufficient data have been gathered. Guest et al. (2006) assert that specific tools are required to determine if the sample size is sufficient for analysis. 1.4 Monte Carlo simulations to verify data saturation A technique that has been proposed to determine if the sample size of a qualitative survey is adequate, is to use the data so obtained and subject it to Monte Carlo simulations (Tran et al., 2016). The resampling of the data provides a technique to assess objectively the cumulative number of themes identified as a function of the sample size. It was asserted that “data saturation was reached in simulations where at least 90% of all themes coded in the http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 2002021 39(3): 200-213 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 Perspectives in Education 2021: 39(3) original study had been identified” (Tran et al., 2016: 90). Frequent and infrequent themes were identified. A frequent theme was defined as one that had been mentioned by more than 2.5% of the participants. The probability of identifying a new theme as a function of the sample size, as well as the probability of missing an important (frequent) theme, were reported. This method is not only objective, but also easily reproducible, which makes this transparent technique very appealing. It was therefore employed in the study reported here. Since it is not a well-known method and has never been used in statistical educational research before, it was decided to provide the reader with a little more background explaining the method. 2. METHODS 2.1 Participants The target population of this study was undergraduate male and female students enrolled for an introductory statistics module at a South African university. The survey was distributed online and 284 students voluntarily completed it. The sample included 81% black students, 11% white, 6% mixed race and 2% Indian students. This was a true reflection of the racial composition at the university where the study was undertaken. Most of the respondents (74%) were females, which corresponded to the gender distribution of the students enrolled for introductory statistics modules. 2.2 Data gathering It was decided to use an online questionnaire because students are often afraid to speak out, especially against their lecturers, for fear of being victimised. Such a study is completely anonymous and participants can write down anything about the module/lecturer/study material, without fear of being labelled or victimised afterwards. 2.3 Data analysis The open-ended questions were analysed using Atlas TI©. Codes were assigned to data fragments after which the codes were synthesised. Thematic analysis was used to explore the data to identify, analyse and report patterns within the data (Clarke & Braun, 2018). Themes for each question were identified, which were then divided into two groups: those that occurred frequently, i.e. more than 2.5% of the participants mentioned them, as defined by Tran et al. (2016), and those that reported occasionally, i.e. by less than 2.5% of the participants. To determine if data saturation was reached with the number of participants using the open- ended questionnaire design, the Monte Carlo simulation method proposed by Tran et al. (2016) was followed. The resampling of the data provided the opportunity to assess the cumulative number of themes identified as a function of the number of participants in the study population. In accordance with Tran et al. (2016), data saturation was defined as having been reached when 90% or more of all the themes coded in the original data set had been identified. Simulations were repeated 10 000 times. Upward coding was then used on the themes that were reported frequently. The aim of this was to move towards an advanced level of abstraction, as explained by Jansen (2010). The ideas that emerged after analysing the voice of the students qualitatively were then compared to the recommendations of the GAISE report to determine if students studying in South Africa raised similar issues, or if they had ideas that differed substantially. http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 2012021 39(3): 201-213 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 Coetzee Determining the needs of introductory statistics university students 3. RESULTS 3.1 Enjoyment Of the 284 respondents, only 144 replied to the statement: “I will enjoy statistics more if…” The researcher identified 27 different themes of which 15 occurred frequently (more than 2.5% of the participants mentioned them [Tran et al., 2016]), and 12 arose occasionally (they were reported by less than 2.5% of the participants). Overall, 84% of the respondents used frequent themes. The simulation was run 10 000 times. The percentage of frequent themes (indicating what would enable a student to enjoy statistics more) that was covered, is indicated in Figure 1, according to the sample size that was taken. At a sample size of 100, 99% of the frequent themes were reported and 93% of all the themes. This indicates that the 144 respondents who answered the question are more than adequate to include all the frequent themes. It is highly unlikely that a frequent theme was missed. Figure 1: Percentage of enjoyment themes covered. The probability of a theme being excluded from a data set, according to sample size, is indicated in Figure 2. At a sample size of 20, 38% of the frequent themes were excluded, and 52% of all the themes. As the sample size increased, these probabilities dropped. When considering samples of size 100, only 0.2% of the frequent themes was excluded and 6.5% of all the themes. http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 2022021 39(3): 202-213 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 Perspectives in Education 2021: 39(3) Figure 2: Probability of an enjoyment theme being excluded from the data set When considering the percentage of themes covered as well as the probability of a theme being excluded from the data set, it can be seen from Figure 1 and Figure 2 that a sample size of 160 would have been sufficient. However, we used a much bigger sample of 284 respondents, therefore we can be confident that our sample was sufficient to achieve data saturation. The most common response (13% of the returns) was that students wanted an active learning experience. They indicated that they desired fun activities, more interaction with the lecturer and with fellow students, more computer-based problems and more challenges. They required more practical sessions where they get an opportunity to practise in class. A few of the responses received are provided below. …if there were more fun activities included in the work. …it included some fun or laughs in between lectures. …more examples are produced with the content as my understanding is built on practicals, doing it. …it was more practical and it applied to daily activities The students indicated that they would enjoy statistics more if they understood it better. Overall, 7.9% of the respondents specified that if statistics were easier to understand and less confusing, they would enjoy it more, whereas 7.2% of the students indicated that they would enjoy statistics more if the content of the course was changed. Not all the students wanted it to be changed in the same way, though. Some would have liked more calculations and less theory. Others preferred more theory. A few wanted fewer formulae. http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 2032021 39(3): 203-213 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 Coetzee Determining the needs of introductory statistics university students Assessment was another frequent theme. Some preferred more multiple-choice questions, others would have liked an open-book test. One undergraduate proposed that the exam should count less than the tests. Students wanted to understand the relevance of statistics to their future career. They desired to know how it is related to what they might do one day in the workplace. Some of the responses are provided below. …if we did practicals of how statistics is used in the workplace pertaining to the course I’m studying. …if it was related to what I will be doing in future. …if it had real-life problems. …if the lecturer could try to be a bit slow while teaching us because some of us think she’s too fast. …it is related to new developments in the real world. The statistics lecturer was another recurrent theme. They wanted a lecturer who is competent and can explain clearly, one that can link new work to previous knowledge and a person who can speak the language of tuition well. …if I had a good lecturer who was patient enough to help me understand, and if the lecturers changed their teaching ways. In addition to this, respondents indicated that assistants would make statistics more enjoyable. Table 1 indicates all the frequent themes that were mentioned for greater enjoyment. Table 1: Frequent enjoyment themes identified during 10 000 simulations Frequent theme for enjoyment Frequency Per cent Cumulative Per cent Active learning 20 13.2 13.2 Understand better 12 7.9 92.1 Content 11 7.2 42.8 Visual learning 11 7.2 99.3 Assessment 9 5.9 23.0 Relevance to degree/job 9 5.9 75.0 Lecturer–explain 8 5.3 51.3 Maths flare 8 5.3 63.2 Assistance 7 4.6 27.6 Already enjoy 6 4.0 17.1 Lecturer–slower 5 3.3 54.6 Less complicated 5 3.3 57.9 Attend classes 4 2.6 30.3 Memos 4 2.6 65.8 Time management 4 2.6 84.2 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 2042021 39(3): 204-213 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 Perspectives in Education 2021: 39(3) 3.2 Performance Of the 284 respondents only 144 gave responses to the statement: “I will perform better in statistics if…” We identified 25 different themes of which 12 occurred frequently, and 13 arose occasionally. Of the respondents, 84% used frequent themes. The simulation was run 10 000 times. Since 49% of the data were missing values, it was decided to include the missing values in the population from which the samples were drawn in the simulation, to see when saturation was reached in such a sparse data set. The percentage of themes (indicating what would enable a student to perform better in statistics) that was covered, is indicated in Figure 3, according to the sample size. At a sample size of 20, only 47% of the frequent themes were produced and 33% of all the themes. Increasing the sample size to 160, 99% of the frequent themes were elicited and 86% of all the themes. Figure 3: Percentage of the performance themes covered. The probability of a theme being excluded from a data set, according to sample size, is indicated in Figure 4. At a sample size of 20, 52% of the frequent themes were excluded and 67% of all the themes. As the sample size increased, these probabilities dropped. At a sample size of 160, only 1% of the frequent themes was excluded and 14% of all the themes. http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 2052021 39(3): 205-213 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 Coetzee Determining the needs of introductory statistics university students Figure 4: Probability of a performance theme being excluded from the data set. Bearing in mind the percentage of themes covered, together with the probability of a theme being excluded from the data set, it can be appreciated from Figures 3 and 4 that a sample size of 250 would have been sufficient. Our sample of 284 respondents is satisfactorily greater than 250, therefore data saturation can be assumed to have been reached. The greatest proportion of students (17.5%) indicated that they would perform better in statistics if they studied more. They realised that they should do their homework, prepare before class, ask questions and put in much more time. Another common theme was time management. The students declared that they should manage their time better, spend more time studying and practising and studying in advance. They should allow themselves more time to study statistics in general. Active learning was another popular topic for a better performance. The students indicated that they would perform better if they had more practical computer work, if there were more class activities and if games were used to teach statistics. … if it is more practical (on computers etc.) to show us how we will use it one day. It will make it easier for us. Practising statistics was mentioned several times. The students said that they needed more practice opportunities and had to spend more time practising. Many students believed that they would have performed better if they had a flair for mathematics. They said things like: “if I were a genius”, “if I was good at maths”, “if I enjoyed working with numbers”, and “if I didn’t need a mathematical background”. http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 2062021 39(3): 206-213 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 Perspectives in Education 2021: 39(3) Assessment was once again a frequent theme. Students would like more small tests often, so that they can study a small section of the work, write and receive constructive feedback. Some students indicated that their achievement would improve if the lecturer explained complicated subject matter better, broke down study units into manageable parts, changed teaching methods and did not become emotional if students did not understand the work. Table 2 indicates the frequent themes mentioned for better performance. Table 2: Frequent performance themes identified during 10 000 simulations. Frequent theme for better performance Frequency Per cent Cumulative per cent Study more 25 17.5 17.5 Time management 22 15.4 32.9 Active learning 12 8.4 41.3 Practice more 11 7.7 48.9 Maths flare 10 7.0 55.9 Assessment 9 6.3 62.2 Lecturer 9 6.3 68.5 Assistance 5 3.5 72.0 Attitude 5 3.5 75.5 Content 4 2.8 78.3 Train to answer 4 2.8 81.1 Understand better 4 2.8 83.9 4. DISCUSSION AND RECOMMENDATIONS The Monte Carlo simulations performed on the results indicated that data saturation was satisfactorily obtained. This study therefore serves as an example of how to use the method in qualitative survey research that records the voice of students and in particular, yielded insight into the students’ perceptions of what they would enjoy in the statistics class and what they think would improve their performance. Many of the themes that emerged can be linked to recommendations made in the GAISE report. Most of the frequent themes can be grouped under faculty, student or lecturer (see Figure 5). The topics that need to be addressed by faculty include relevance to degree/job, subject content, focus of the content, the amount of work, the method of assessment and assistance offered to students. http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 2072021 39(3): 207-213 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 Coetzee Determining the needs of introductory statistics university students Figure 5: A graphical representation of the themes, grouped in terms of the three principal influences. The student group indicated that they need to appreciate the relevance of statistics to their future career. Many felt they did not understand why they should spend time and energy on something that they did not think would ever be used. One way to address this is to ensure that the examples employed are directly linked to the degree course. A psychology student should have examples and exercises related to the field of psychology, whereas a business student should rather have examples directed at how statistics can assist a business owner to run his company more profitably. This can be addressed by teaching the investigative process of problem-solving and decision-making, exposing the students to real data with a context and purpose and analysing data that relate to their field of study, as advocated in the GAISE college report (Wood et al., 2018). Many of the themes emerging from the student voice are addressed by the GAISE report. Figure 6 shows which themes are addressed and which themes are not directly addressed, although some of them may be indirectly appealed to. http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 2082021 39(3): 208-213 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 Perspectives in Education 2021: 39(3) Figure 6: Student voice: topics addressed by the GAISE report, and those not addressed. It was also clear that there is a difference in the focus of the content among students. The one-size-fits-all approach of current introductory modules does not consider the different needs of students from a wide variety of fields. Faculty needs to determine what the particular needs of a specific group of students are and develop a tailor-made module. The amount of work that should be covered was another popular theme that emerged. For some fields, less may be better. For students who are not comfortable with numbers and calculations, it may be sensible to include less study material in a module, and to ensure that the focus with regard to practical versus theoretical will meet the desired needs. Giddens and Brady (2007) refer to “content saturation”, which is characterised by curricula being overloaded with content. This results in covering a wide range of topics but leads to a lack of deep understanding. Critical thinking is also not being developed. Burch et al. (2015) suggest the use of a conception-based curriculum to overcome these problems. The GAISE report endorses that if the focus were more on teaching statistical thinking and conceptual understanding by concentrating more on the process of solving problems and making decisions, rather than emphasising monotonous calculations, a better understanding could be achieved (Wood et al., 2018). Students indicated that the method of assessment should be addressed. A practical project may be better. Formative assessments where smaller pieces of work are covered, followed by constructive feedback, are requested. This issue was also addressed in the GAISE report, which noted that assessments should not only be used to evaluate student learning, but also to improve it (Wood et al., 2018). It is essential to transform the way students solve problems to move away from the mechanical use of formulae. They should get to the point where they want to understand the concepts, not only to be able to repeat exercise solutions, but to solve real problems. This can be accomplished by using project-based instruction (Magalhães & Magalhães, 2014). http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 2092021 39(3): 209-213 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 Coetzee Determining the needs of introductory statistics university students In some fields, it is important that students understand the formulae and are able to do the calculations. Burch et al. (2015) advocate the conception-focused curriculum to introduce statistics students to the “real world”. Students who were exposed to this curriculum indicated that they experienced deeper learning than with a traditional curriculum. Faculty also needs to provide the means by which assistants can be made available to students should they require it, as proposed by Crowe, Ceresola and Silva (2014), who recommend the use of undergraduate teaching assistants, because students’ performances improve significantly when they are helped by peers. Students realised that they themselves are responsible for their enjoyment of, and performance in, the subject. They indicated that they would enjoy it more if they managed their time better. They acknowledged that they needed to put in much more work. Extensive research has indicated the positive correlation between time management and performance (George et al., 2008; Häfner, Oberst & Stock, 2014; Häfner, Stock & Oberst, 2015; Meeuwisse, Born & Severiens, 2013). Since 81% of our survey sample were Africans, whose African cultures tend to let issues emerging from the immediate future dictate their time use, African students may well not place a high value on long-term planning (Mpofu et al., 1996). This should be kept in mind if faculty decides to develop a programme to train undergraduates in time management. On the other hand, some participants thought that since they do not have a flair for mathematics, they were not able to enjoy statistics. This perception may be difficult to change. If the focus of the module moves from calculations to conceptual understanding and interpreting, students may realise that they can succeed in applying statistics even with a somewhat compromised mathematical foundation. In addition, the students indicated that they would probably perform better if they had a more positive attitude towards statistics, which reflected research on the correlation between attitude towards and performance in statistics (Nguyen et al., 2016; Ratanaolarn, 2016). A negative attitude may change if the module’s content is more applicable to the student’s field of study and if the lecturer presents it in a manner that makes it more enjoyable and effective. Respondents indicated that their classroom experience had a great effect on their enjoyment of statistics and that they saw the lecturer as the person ablest to improve this. Students do not enjoy a passive classroom where they simply receive the content. They enjoy it more when they are actively involved. Rowe et al. (2015) suggested that, by providing students with challenging assignments and opportunities to relate their learning to real life, their emotions will become more positive towards their studies. However, it is not an easy task to implement active learning successfully (Strayer et al., 2019). A method that can be used to ensure that students are actively involved, is to use a flipped classroom where the class use online slides and/or videos to learn the concepts and then class time is utilised for interactive group learning. Swart and Wuensch (2016) used this approach in a quantitative business class and found that it increased student satisfaction and retention while keeping academic standards high. Exploring statistical concepts and analysing data using technology is another way to involve students. Students are no longer content with boring presentations (slides) with many formulae and calculations. They want a visual experience. They want to see how different concepts are linked to one another. Research conducted by Chei-Chang (2009) and Chiou et al. (2017) http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 2102021 39(3): 210-213 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 Perspectives in Education 2021: 39(3) indicate that concept mapping can be used effectively in the statistics classroom. Construct- on-scaffold concept mapping whereby students are provided with a basic concept map structure, which they have to complete, may be a good way to get students actively involved in a class and guide them to find the links between different concepts for themselves, and presenting it in a visually attractive manner. Students would also like colourful study guides and supplemental videos to help them understand better. Here a lecturer may provide a platform where students themselves can upload links to videos they find helpful. Our students indicated that a statistics lecturer should be someone who has the ability to explain difficult concepts in an easy way. The lecturer should be in touch with the students to know when to slow down, repeat or explain from another angle. An ideal lecturer is one who has high emotional stability and conscientiousness, and likes to be among people (Kim & MacCann, 2016). The themes that emerged from the students’ responses indicate that it will be worthwhile for South African universities to implement the recommendations of the GAISE college report, since the voice of the students echoed the needs addressed in this document. While the report does not directly address all the issues raised by the students, it is possible that it may have an indirect influence; for example, if students understand the relevance of statistics by applying technology and the investigative process to real data to find solutions that relate to their field of study, their attitude towards statistics may improve and they may be more motivated to put more time and effort into their statistical studies. 5. FUTURE RESEARCH The research reported here included students from one university only. To generalise the findings, similar research should be conducted at other institutions. The qualitative study should be complemented by a quantitative one to determine which particular factors are associated with greater enjoyment of, and better performance in, statistics. More qualitative survey research studies should employ the Monte Carlo simulation method to determine if data saturation has been reached. 6. CONCLUSION The use of Monte Carlo simulations was found to be useful to determine if data saturation had been reached in this qualitative survey study. One of the benefits of this method is that studies could be objectively compared if this approach was used. This study produced valuable insight into students’ experiences of their introductory statistics modules. Topics that need to be addressed, as identified by students studying at a South African university, are similar to recommendations of the US-based GAISE college report (Wood et al., 2018). Students did not want to learn for the sake of learning, they desired to know that they will be able to apply their knowledge in their field of work one day. It is therefore imperative that faculties should think critically about the way they structure their introductory statistics modules. The content, its focus as well as the amount of work to be covered should be carefully planned to be relevant to the field in which the student is likely to work one day and prepare them to be comfortable in a data-rich world. Assessment methods should be used http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 2112021 39(3): 211-213 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 Coetzee Determining the needs of introductory statistics university students that encourage critical thinking and understanding rather than mechanical reproduction of calculations. Creating platforms for peer assistance would be appreciated by students. Undergraduates should be guided to manage their time better to enable them to study and practise more. Contact sessions must be fun with lots of opportunities for active learning so that students will want to attend class. By making the content more appropriate and the learning experience more worthwhile, students’ attitudes towards statistics may improve. Moreover, they need statistics lecturers who are able to explain difficult concepts in an understandable way and are able to adapt their teaching style and speed of delivery to the specific group they are lecturing to. When these suggestions are implemented, the students’ enjoyment as well as their performance in statistics modules should both increase. This should better prepare the graduates for the challenging world of the fourth industrial revolution. 7. ACKNOWLEDGEMENTS I thank North-West University for allowing me to conduct this study. 8. DECLARATION No potential conflict of interest was reported by the author. REFERENCES Burch, G.F., Burch, J.J., Heller, N.A. & Batchelor, J.H. 2015. An empirical investigation of the conception focused curriculum: The importance of introducing undergraduate business statistics students to the ‘real world’. Decision Sciences Journal of Innovative Education, 13: 485–512. https://doi.org/10.1111/dsji.12074 Chei-Chang, C. 2009. Effects of concept mapping strategy on learning performance in business and economics statistics. Teaching in Higher Education, 14: 55–69. https://doi. org/10.1080/13562510802602582 Chiou, C.-C., Lee, L.-T., Tien, L.-C. & Wang, Y.-M. 2017. Analyzing the effects of various concept mapping techniques on learning achievement under different learning styles. EURASIA Journal of Mathematics, Science & Technology Education, 13: 3687–3708. https:// doi.org/10.12973/eurasia.2017.00753a Clarke, V. & Braun, V. 2018. Using thematic analysis in counselling and psychotherapy research: A critical reflection. Counselling & Psychotherapy Research, 18(2): 107–110. https:// doi.org/10.1002/capr.12165 Cook-Sather, A. 2002. Authorizing students’ perspectives: Toward trust, dialogue, and change in education. Educational Researcher, 31: 3–14. https://doi.org/10.3102/0013189X031004003 Crowe, J., Ceresola, R. & Silva, T. 2014. Enhancing student learning of research methods through the use of undergraduate teaching assistants. Assessment & Evaluation in Higher Education, 39: 759–775. https://doi.org/10.1080/02602938.2013.871222 Dunn, K.E., Lo, W.-J., Mulvenon, S.W. & Sutcliffe, R. 2012. Revisiting the motivated strategies for learning questionnaire: A theoretical and statistical reevaluation of the metacognitive self- regulation and effort regulation subscales. Educational and Psychological Measurement, 72: 312–331. https://doi.org/10.1177/0013164411413461 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 https://doi.org/10.1111/dsji.12074 https://doi.org/10.1080/13562510802602582 https://doi.org/10.1080/13562510802602582 https://doi.org/10.12973/eurasia.2017.00753a https://doi.org/10.12973/eurasia.2017.00753a https://doi.org/10.1002/capr.12165 https://doi.org/10.1002/capr.12165 https://doi.org/10.3102/0013189X031004003 https://doi.org/10.1080/02602938.2013.871222 https://doi.org/10.1177/0013164411413461 2122021 39(3): 212-213 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 Perspectives in Education 2021: 39(3) George, D., Dixon, S., Stansal, E., Gelb, S L. & Pheri, T. 2008. Time diary and questionnaire assessment of factors associated with academic and personal success among university undergraduates. Journal of American College Health, 56: 706–715. https://doi.org/10.3200/ JACH.56.6.706-715 Giddens, J.F. & Brady, D.P. 2007. Rescuing nursing education from content saturation: The case for a concept-based curriculum. Journal of Nursing Education, 46: 65–69. https://doi. org/10.3928/01484834-20070201-05 Guest, G., Bunce, A. & Johnson, L. 2006. How many interviews are enough? An experiment with data saturation and variability. Field Methods, 18: 59–82. https://doi. org/10.1177/1525822X05279903 Häfner, A., Oberst, V. & Stock, A. 2014. Avoiding procrastination through time management: An experimental intervention study. Educational Studies, 40: 352–360. https://doi.org/10.108 0/03055698.2014.899487 Häfner, A., Stock, A. & Oberst, V. 2015. Decreasing students’ stress through time management training: An intervention study. European Journal of Psychology of Education, 30: 81–94. https://doi.org/10.1007/s10212-014-0229-2 Hanson, J.L., Balmer, D.F. & Giardino, A.P. 2011. Qualitative research methods for medical educators. Academic Pediatrics, 11: 375–386. https://doi.org/10.1016/j.acap.2011.05.001 Jaafar, Z., Malek, H.A., Kheng, F.K. & Ali, R. 2012. Cognitive factors influencing statistical performance of diploma science students: A structural equation model approach. International Conference on Statistics in Science, Business and Engineering (ICSSBE), 1–5. https://doi. org/10.1109/ICSSBE.2012.6396628 Jansen, H. 2010. The logic of qualitative survey research and its position in the field of social research methods. Forum: Qualitative Social Research, 11: 1–21. Kennedy, T.J.T. & Lingard, L.A. 2006. Making sense of grounded theory in medical education. Medical Education, 40: 101–108. https://doi.org/10.1111/j.1365-2929.2005.02378.x Kim, L.E. & Maccann, C. 2016. What is students’ ideal university instructor personality? An investigation of absolute and relative personality preferences. Personality and Individual Differences, 102: 190–203. https://doi.org/10.1016/j.paid.2016.06.068 Magalhães, M.N. & Magalhães, M.C.C. 2014. A critical understanding and transformation of an introductory statistics course. Statistics Education Research Journal, 13: 28–41. https:// doi.org/10.52041/serj.v13i2.278 Meeuwisse, M., Born, M.P. & Severiens, S.E. 2013. Academic performance differences among ethnic groups: Do the daily use and management of time offer explanations? Social Psychology of Education, 16: 599–615. https://doi.org/10.1007/s11218-013-9231-9 Mega, C., Ronconi, L. & De Beni, R. 2014. What makes a good student? How emotions, self-regulated learning, and motivation contribute to academic achievement. Journal of Educational Psychology, 106: 121–131. https://doi.org/10.1037/a0033546 Morris, P.E. & Fritz, C.O. 2015. Conscientiousness and procrastination predict academic coursework marks rather than examination performance. Learning and Individual Differences, 39: 193–198. https://doi.org/10.1016/j.lindif.2015.03.007 Mpofu, E., D’amico, M. & Cleghorn, A. 1996. Time management practices in an African culture: Correlates with college academic grades. Canadian Journal of Behavioural http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 https://doi.org/10.3200/JACH.56.6.706-715 https://doi.org/10.3200/JACH.56.6.706-715 https://doi.org/10.3928/01484834-20070201-05 https://doi.org/10.3928/01484834-20070201-05 https://doi.org/10.1177/1525822X05279903 https://doi.org/10.1177/1525822X05279903 https://doi.org/10.1080/03055698.2014.899487 https://doi.org/10.1080/03055698.2014.899487 https://doi.org/10.1007/s10212-014-0229-2 https://doi.org/10.1016/j.acap.2011.05.001 https://doi.org/10.1109/ICSSBE.2012.6396628 https://doi.org/10.1109/ICSSBE.2012.6396628 https://doi.org/10.1111/j.1365-2929.2005.02378.x https://doi.org/10.1016/j.paid.2016.06.068 https://doi.org/10.52041/serj.v13i2.278 https://doi.org/10.52041/serj.v13i2.278 https://doi.org/10.1007/s11218-013-9231-9 https://doi.org/10.1037/a0033546 https://doi.org/10.1016/j.lindif.2015.03.007 2132021 39(3): 213-213 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 Coetzee Determining the needs of introductory statistics university students Science / Revue canadienne des sciences du comportement, 28: 102–112. https://doi. org/10.1037/0008-400X.28.2.102 Nguyen, T.H., Charity, I. & Robson, A. 2016. Students’ perceptions of computer-based learning environments, their attitude towards business statistics, and their academic achievement: Implications from a UK university. Studies in Higher Education, 41: 734–755. https://doi.org/1 0.1080/03075079.2014.950562 Putwain, D.W., Becker, S., Symes, W. & Pekrun, R. 2018. Reciprocal relations between students’ academic enjoyment, boredom, and achievement over time. Learning and Instruction, 54: 73–81. https://doi.org/10.1016/j.learninstruc.2017.08.004 Ramani, S. & Mann, K. 2016. Introducing medical educators to qualitative study design: Twelve tips from inception to completion. Medical Teacher, 38: 456–463. https://doi.org/10.3 109/0142159X.2015.1035244 Ratanaolarn, T. 2016. The development of a structural equation model of graduate students’ statistics achievement. International Journal of Behavioral Science, 11: 153–168. Rowe, A.D., Fitness, J. & Wood, L.N. 2015. University student and lecturer perceptions of positive emotions in learning. International Journal of Qualitative Studies in Education (QSE), 28: 1–20. https://doi.org/10.1080/09518398.2013.847506 Schukajlow, S. & Rakoczy, K. 2016. The power of emotions: Can enjoyment and boredom explain the impact of individual preconditions and teaching methods on interest and performance in mathematics? Learning and Instruction, 44: 117–127. https://doi.org/10.1016/j. learninstruc.2016.05.001 Starks, H. & Trinidad, S.B. 2007. Choose your method: A comparison of phenomenology, discourse analysis, and grounded theory. Qualitative Health Research, 17: 1372–1380. https://doi.org/10.1177/1049732307307031 Strayer, J.F., Gerstenschlager, N.E., Green, L.B., Mccormick, N., Mcdaniel, S. & Rowell, G.H. 2019. Toward a full(er) implementation of active learning. Statistics Education Research Journal, 18(1): 63–82. https://doi.org/10.52041/serj.v18i1.150 Swart, W. & Wuensch, K.L. 2016. Flipping quantitative classes: A triple win. Decision Sciences Journal of Innovative Education, 14: 67–89. https://doi.org/10.1111/dsji.12088 Tavakol, M. & Sandars, J. 2014. Quantitative and qualitative methods in medical education research: Amee guide no 90: Part ii. Medical Teacher, 36: 838–848. https://doi.org/10.3109/0 142159X.2014.915298 Tran, V.-T., Porcher, R., Falissard, B. & Ravaud, P. 2016. Point of data saturation was assessed using resampling methods in a survey with open-ended questions. Journal of Clinical Epidemiology, 80: 88–96. https://doi.org/10.1016/j.jclinepi.2016.07.014 Warren, C.A. & Marciano, J.E. 2018. Activating student voice through youth participatory action research (ypar): Policy-making that strengthens urban education reform. International Journal of Qualitative Studies in Education, 31: 684–707. https://doi.org/10.1080/09518398. 2018.1478154 Wood, B.L., Mocko, M., Everson, M., Horton, N.J. & Velleman, P. 2018. Updated guidelines, updated curriculum: The gaise college report and introductory statistics for the modern student. Chance, 31: 53–59. https://doi.org/10.1080/09332480.2018.1467642 http://dx.doi.org/10.18820/2519593X/pie.v39.i3.15 https://doi.org/10.1037/0008-400X.28.2.102 https://doi.org/10.1037/0008-400X.28.2.102 https://doi.org/10.1080/03075079.2014.950562 https://doi.org/10.1080/03075079.2014.950562 https://doi.org/10.1016/j.learninstruc.2017.08.004 https://doi.org/10.3109/0142159X.2015.1035244 https://doi.org/10.3109/0142159X.2015.1035244 https://doi.org/10.1080/09518398.2013.847506 https://doi.org/10.1016/j.learninstruc.2016.05.001 https://doi.org/10.1016/j.learninstruc.2016.05.001 https://doi.org/10.1177/1049732307307031 https://doi.org/10.52041/serj.v18i1.150 https://doi.org/10.1111/dsji.12088 https://doi.org/10.3109/0142159X.2014.915298 https://doi.org/10.3109/0142159X.2014.915298 https://doi.org/10.1016/j.jclinepi.2016.07.014 https://doi.org/10.1080/09518398.2018.1478154 https://doi.org/10.1080/09518398.2018.1478154 https://doi.org/10.1080/09332480.2018.1467642