Research in Social Sciences and Technology https://ressat.org E-ISSN: 2468-6891 Volume: 8 Issue: 3 2023 pp. 16-35 Framework for Numeracy and Digital Skills Attributes in Higher Education Siti Fairuz Dalima, Sharipah Ruzaina Syed Aris*a, Teoh Sian Hoona, Fazyudi Ahmad Nadzria, Sayang Mohd Denib, Norasikin Yahyac, & Ellianawati M.Sid * Corresponding author Email: sruzaina@uitm.edu.my a. Faculty of Education, Universiti Teknologi MARA, Puncak Alam, Malaysia b. College of Computing, Informatics and Media, Universiti Teknologi MARA, Shah Alam, Malaysia c. Quality Assurance Training Centre, Malaysian Qualification Agency, Cyberjaya, Malaysia d. UNNES, Indonesia Article Info Received: April 11, 2023 Accepted: June 13, 2023 Published: August 12, 2023 How to cite Dalim, S. F., Aris, S. R. S., Hoon, T. S., Nadzri, F. A., Deni, S. M., Yahya, N., & Si, E. M. (2023). Framework for numeracy and digital skills attributes in higher education. Research in Social Sciences and Technology, 8(3), 16-35. https://doi.org/10.46303/ressat.2023.18 Copyright license This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International license (CC BY 4.0). ABSTRACT Numeracy and digital skills are among the most crucial skills that any graduate should possess before entering the workforce, regardless of their field of study. Therefore, both skills should not be treated as an “add on” skill but viewed as essential graduate attributes needing to be purposefully incorporated into the curriculum instead of mere chance. By understanding the learners’ numeracy and digital skills abilities, educators can create suitable learning outcomes, activities, and assessments, enabling them to acquire the skills. The objective of this study is to create a framework for testing numeracy (NSI) and digital skills (DSI) using a cross-sectional design and quantitative methodology. The research entails generating a questionnaire, verifying it via a rigorous procedure of expert content validation, and focusing on sufficiency, clarity, coherence, and relevance. A pilot study involving 218 students from various disciplines was conducted to measure the reliability of the framework using Cronbach Alpha. Feedback from experts was then used to enhance the questionnaire, resulting in the finalisation of five components for the Digital Skills Instrument (DSI): information literacy, computer and technology literacy, digital communication and collaboration skills, digital identity and well-being, and digital ethics. Meanwhile, the five components of the Numeracy Skills Instrument (NSI) encompassed operation and calculation, graphical representation, quantitative reasoning and logical thinking, complex number (advanced concept), spatial visualisation and geometric reasoning. The results indicated a strong internal consistency across all components for both tools, with alpha values ranging from 0.847 to 0.958 for DSI and 0.916 to 0.964 for NSI. Corrected item-total correlations also depicted intercorrelation between items for both instruments. To sum up, the findings demonstrate that the DSI and NSI have high reliability and validity. Because both tools are reliable and valid for measuring digital and numeracy skills in the Malaysian context, they can be confidently used for future research. KEYWORDS Numeracy skills; digital skills; graduate attributes; framework; higher education. 10.46303/ ressat.2023.18 mailto:sruzaina@uitm.edu.my https://doi.org/10.46303/ressat.2023.18 17 RESSAT 2023, 8(3): 16-35 INTRODUCTION Several studies have depicted the positive impact of numeracy and digital literacy. For example, The Organization for Economic Cooperation and Development (OECD, 2013) performed research and reported a significant association between numeracy skills and levels of educational achievement, employment, and improved health outcomes and life satisfaction. Moreover, many studies have also accentuated the importance of skills for future survival (Aris et al., 2022; Kee et al., 2023; Vodă et al., 2022). Similarly, previous European Commission (2015) research determined that digital skills are essential for personal and professional development and are in rising demand in the job market. Numeracy and digital skills have received increased attention in today's rapidly changing technological world. Numeracy encompasses the ability to effectively apply mathematical concepts and methodologies in daily life, including interpreting and using numerical information and data. Digital skills entail using digital technologies, such as computers, the internet, and mobile devices, to access, process, and communicate information. These talents’ value and influence on individual and societal well-being have become more visible in recent years. This situation has resulted in greater investment in programs and activities to enhance numeracy and digital literacy at all societal levels. The widespread acceptance of the importance of numeracy and digital skills is reflected in their inclusion in national and international frameworks for education and skills development. For example, the European Qualifications Framework (EQF) includes both numeracy and digital skills as key competencies (European Commission, 2008). Similarly, the Malaysian Qualifications Framework (MQF) recognizes the importance of these skills and has implemented them as learning outcomes at all levels of education (Ministry of Higher Education, Malaysia, 2018). Despite this recognition, a critical gap exists between the supply and demand for numeracy and digital skills, particularly in developing countries. A World Bank (2015) study found that despite progress in improving digital skills, substantial disparities still exist between different countries and regions. Numeracy and digital abilities are usually considered crucial for personal and professional growth, and their demand in the worldwide job market has risen. Because technology immensely impacts our lives, continual investment in programs and activities is vital to enhance these abilities. Consequently, this process can assist in bridging the gap between supply and demand and contribute to a more skilled, inventive, and inclusive society. Comprehending the importance of both skills, a team of academics initiated the framework of numeracy and digital skills in the Malaysian environment. Thanks to this research, all curriculum developers in higher education are expected to recognise their students' numeracy and digital abilities and construct curricula that may boost future skill sets. It can only be realized by developing reliable and valid instruments for measuring numeracy (NSI) and digital skills (DSI), based on the conceptual definitions outlined in the Malaysian Qualifications Framework (MQA, 2018). A preliminary study conducted in a Malaysian university had students from various fields. 18 Dalim et al RESSAT 2023, 8(3): 16-35 The proposed framework can prove a valuable resource for curriculum developers to support the development of their students’ numeracy and digital skills. Ultimately, our findings will stimulate the ongoing efforts to upgrade the quality of higher education in Malaysia and enhance the employability of graduates in an increasingly digital and technologically driven competitive world. Numeracy and Digital Attributes as Future skill set Digital and numeracy skills are crucial for success in today's rapidly evolving technological world. With technological advancements continuously modifying how we live and work, possessing strong digital and numeracy skills is essential. Studies have revealed that these skills are positively related to job performance, and their importance has increased in the modern workforce. Kim et al. (2016) indicated that numeracy skills are positively associated with job performance and have become an indispensable part of the contemporary workforce owing to technological advances. Individuals wishing to compete in an increasingly data-driven industry must possess superb numeracy abilities. Numeracy skills can involve an individual's ability to understand, use, and apply mathematical concepts and techniques in real-world situations. The term encompasses various skills: computation, estimation, problem-solving, and data analysis. Numeracy skills are considered a critical component of overall literacy and can bring imminent success in many sectors, including education, employment, and financial management. The United Nations Educational, Scientific and Cultural Organization (UNESCO) defined numeracy as "the capacity to identify, understand, interpret, create, communicate, and use mathematical concepts and structures to solve problems and make informed decisions." This definition highlights the importance of numeracy skills in solving mathematical problems and making informed decisions in various contexts. In recent years, numeracy skills have been widely acknowledged as a crucial set of abilities for the future. Numeracy skills, encompassing mathematical literacy, involve the capacity to comprehend and utilize mathematical concepts in everyday situations. With the advent of technology and the prevalence of data-driven decision-making, numeracy skills have gained heightened importance across various domains, such as finance, engineering, healthcare, and education. As per a report published by the Organization for Economic Cooperation and Development (OECD), numeracy skills are deemed crucial for individuals to partake actively in society and capitalize on the opportunities arising from technological advancements (OECD, 2019). Yamashita et al. (2023) accentuate the prominence of numeracy skills for STEM and non- STEM professionals because numeracy plays a vital role in various aspects of daily life, including managing personal finances, accessing health care, and engaging in civic engagement. The study further highlights that numeracy skills are valuable for all workers, regardless of their field. Promoting numeracy skills can substantially benefit individuals and society. 19 RESSAT 2023, 8(3): 16-35 Furness et al. (2017) present a compelling argument regarding the relevance of numeracy abilities for educators and students. They assert that a comprehensive understanding of mathematical concepts and critical thinking while solving problems are vital for effective mathematics education. Kars-Tietema and Kars (2018) stress the role of schools in equipping children with robust numeracy skills to prepare them for future occupations. They recommend incorporating real-world challenges and technology into mathematics education to empower students with the necessary skills for success in the twenty-first century. These studies highlight the need to emphasize numeracy skills across all levels of education, including teacher preparation programs and the K-12 curriculum. Similarly, digital skills have gained an unprecedented presence in today's workforce, encompassing computer usage, internet proficiency, and software application knowledge. Vrana (2006) found that digital literacy skills, including proficiency in word processing, spreadsheets, and other software programs, are positively related to job performance. Likewise, Liu (2016) reported that digital literacy skills will even be more critical for success in the 21st century and play a critical role in lifelong learning. The study emphasized the need for individuals to possess strong digital skills to adapt to the rapidly changing technological landscape and effectively seize new opportunities. In the contemporary world, digital literacy has emerged as an essential element due to the indispensability of technology for smooth functioning. Dewi et al. (2021) define digital literacy as the capacity to access, evaluate, and effectively communicate information through various media platforms. Competence in digital skills relies on multiple factors, including technical skills, critical understanding, and communication skills, with a critical understanding of a paramount importance (Dewi et al., 2021). Vodă et al. (2022) assert that the level of education plays a substantial role in the development of digital abilities (2022). Additionally, critical solid thinking enables individuals to perform a more in-depth investigation and assessment of the media material they encounter (Dewi et al., 2021). Conversely, technical skills are associated with the capacity to access and operate media platforms (Dewi et al., 2021). Individuals with advanced technical skills excel in computer usage and the internet navigation, especially in complex tasks. Proficiency in communication skills empowers individuals to interact with others, engage in online activities, and create media content. The rise of digital skills has facilitated the growth of big data analysis, and given the pervasive adoption of decisions based on big data analysis, an imperative need exists to enhance digital skills (Sivarajah et al., 2017) The Indonesian study team (i.e., Rahmawati et al., 2022) highlights the significance of digital skills in the contemporary world. It examines the digital competence of vocational high school teachers in Indonesia. The authors argue that vocational high school teachers must enhance their digital skills via training and that legislators should adopt regulations to facilitate this. They pose that having excellent digital skills is vital in today's employment market because it promotes communication, access to information, problem-solving capabilities, and 20 Dalim et al RESSAT 2023, 8(3): 16-35 convenience. The essay emphasizes the need for prioritizing the development of digital skills for navigating the digital era’s challenges and opportunities. Numeracy skills and digital skills are closely related because both are essential for effective functioning in the modern world. In the digital age, numeracy and digital skills often overlap, as many digital tools and technologies require a strong foundation in numeracy for efficient use. Bredberg's (2020) article explores the role of mathematics and critical thinking in promoting democracy in digital society. The author contends that in the digital era, individuals must possess both critical thinking skills and mathematical proficiency to navigate the intricate realm of information. Bredberg highlights the importance of education in promoting mathematical and critical thinking skills and suggests that individuals can make informed decisions about important social and political issues thanks to these skills. The article also addresses the potential for digital technologies to support the development of these skills through interactive educational tools and online community practices. Overall, Bredberg's article presents a compelling case for the importance of mathematics and critical thinking in promoting democratic engagement and citizenship in the digital age. In conclusion, digital and numeracy skills have become increasingly relevant in the contemporary technological world. Studies have revealed that these skills are positively related to job performance and are essential for success in the modern workforce. Therefore, as technology evolves, it is critical for individuals to possess strong digital and numeracy skills to succeed. Numeracy and Digital Skills as Graduate Attributes in the Curriculum Framework Numeracy and digital skills have become increasingly important in today's digital age, with the technology playing a vital role in our daily lives. Accordingly, a growing interest exists to incorporate these skills as graduate attributes in curriculum frameworks. Incorporating numeracy and digital skills into curriculum frameworks will allow producing graduates knowledgeable in their field of study and simultaneously having the capability to use digital and numerical technologies effectively. This process will help them better prepare for the current and future workplace demands. Many studies have addressed incorporating numeracy and digital skills into curriculum frameworks and discussed the issue thoroughly. Kee et al. (2023) reported that acquiring digital skills positively influenced students' perceived employability. Additionally, they found that the quality of the courses taken by students mediated the relationship between digital skill acquisition and perceived employability. This condition suggests that high-quality courses are a prerequisite for students to develop the digital skills to enhance their employability. The authors conclude that promoting digital skills among youth can positively impact their employability and that the quality of courses teaching digital skills is crucial to ensuring that students can develop the skills required to succeed in the job market. Ismail and Hassan (2019) investigated the importance of digital technology technical abilities for the Fourth Industrial Revolution (Industry 4.0). Industry 4.0, according to the authors, involves 21 RESSAT 2023, 8(3): 16-35 integrating digital technologies into all facets of the economy, and technical capabilities in digital technology are crucial for people and organisations to survive in this era. It also highlights the need for digital skills such as data analysis, programming, and cybersecurity for labour market competitiveness. The writers also accentuate the vitality of firms creating digital strategies to stay competitive and successful in a continually evolving business environment. The article also addresses the influence of Industry 4.0 on institutions of technical and vocational education and training (TVET). The authors suggest that TVET schools must change their curriculum to adapt to the changing requirements of the labour market and equip students with the technical skills demanded by Industry 4.0. Several considerations must be considered to effectively incorporate numeracy and digital skills into curriculum frameworks. Firstly, it is important to recognize that these skills are not static but constantly develop and change as technology advances. Therefore, it is imperative to regularly revise the curriculum to incorporate the most recent advancements and trends in digital and numeracy skills. Another aspect to consider is the need to provide students with diverse opportunities to develop their numeracy and digital skills. This process can include a combination of formal coursework, project-based learning, and hands-on experience with digital and numerical technologies. Providing students with the support and guidance in their skill development is critical and should include access to resources and mentorship from experienced professionals. A need also exists to adopt a flexible and interdisciplinary approach to incorporating numeracy and digital skills into the curriculum. This necessity can involve working with other departments and schools within the university, and with industry partners to ensure that students receive a well-rounded education ready for challenging careers. The Malaysian Qualifications Agency (MQA) revised its 2007 Qualifications Framework (MQF, 2018) to emphasise more on numeracy and digital qualities responding to changes in the educational landscape and the emergence of new talents. The MQA updated the Malaysian Qualifications Framework to ensure higher-quality academic programs offered by higher education providers in Malaysia. This authority established eight domains of learning outcomes in 2007. In the MQF 2nd edition, the learning outcomes are revised and maintained to form five learning outcome clusters with eleven learning outcome attributes. Each academic program should satisfy each cluster and the attributes or characteristics of learning outcomes. The five clusters encompass knowledge and understanding, cognitive skills, functional and work skills, personal and entrepreneurial skills, and ethics and professionalism. Since April 2019, MQA has mandated that all Malaysian universities incorporate this new framework into their curricula. To meet this requirement, educators must assess their students’ numeracy and digital skills and use this information to create materials and assessments to help students develop and improve these skills. Thanks to this process, educators can understand their students better, allowing them to create a curriculum meeting their needs and assisting them in achieving their learning outcomes. 22 Dalim et al RESSAT 2023, 8(3): 16-35 The MQF provides an overall definition and a set of characteristics serving as minimal criteria for higher education institutions to construct their curricula. Digital skills are defined as the ability to use information and digital technology for work and study. They also comprise acquiring and retaining information, processing data, utilising applications for problem-solving and communication, and ethically and responsibly using digital skills. Information literacy, computer and technology literacy, visual literacy, and digital communication and collaboration skills are the four components of this definition. Meanwhile, numeracy includes understanding basic mathematics and symbols relating to statistical techniques (MQA, 2017). Additionally, according to UNESCO (2018) and based on Quick Reference: 5 Clusters of Learning Outcomes MQF2.0 (MOHE-2020), the definition of numeracy has been expanded to include five main components: quantitative reasoning and logical thinking, graphical representation, spatial visualization and geomatic reasoning, operation and calculation, and complex number. Quantitative reasoning and logical thinking are key components of numeracy, and they are defined as the ability to understand, interpret, and use mathematical information in real- world situations. Quantitative reasoning comprises handling information, identifying mathematical information, calculating, estimating, and solving problems using numerical data. This procedure also includes processing data, understanding statistics and probability, interpreting data, drawing conclusions, and determining the reliability and significance of data. Graphical representation is another important aspect of numeracy because it involves representing and interpreting data in graphs, tables, and diagrams. This representation allows individuals to understand complex mathematical information and to make sense of data visually and intuitively. Spatial visualization and geometric reasoning play an important role in numeracy because they encompass the ability to conceptualise and utilise objects in 2D and 3D spaces and to comprehend angles and positions. It encompasses skills such as mentally representing object arrangements, generating and designing objects, and navigating and orienting oneself. Operation and calculation are critical for numeracy because they involve the procedural rules underlying manipulations of whole numbers, decimals, and fractions. The ability to perform basic arithmetic operations such as addition, subtraction, multiplication, and division is crucial in numeracy. Complex numbers, including calculus and advanced calculus, form part of numeracy. This operation involves the ability to perform complex mathematical tasks, such as problem-solving and mathematical reasoning. Individuals with strong numeracy skills can easily perform routine and non-routine complex numerical tasks. In summary, numeracy skills encompass various mathematical concepts and abilities, including quantitative reasoning and logical thinking, graphical representation, spatial visualization and geometric reasoning, operation and calculation, and complex numbers. These skills are essential for success in many aspects of life: education, employment, and financial management. 23 RESSAT 2023, 8(3): 16-35 METHODOLOGY In this study, we employed a cross-sectional design that utilises a quantitative approach to establish a framework for measuring numeracy and digital skills. Specifically, the study involved developing questionnaire items, validating the questionnaire through a rigorous process of expert content validation, and assessing the internal reliability of the questionnaire using the Cronbach Alpha statistical method. Eleven lecturers from various backgrounds with a minimum of 15 years of teaching and research experience were chosen as content experts in this study to validate the questionnaire. Table 3 depicts the characteristics of the content experts who participated in the study. Meanwhile, for the pilot study, 218 students from various academic disciplines, including non- science, business management, and science and technology fields, voluntarily participated in the pilot study. Numeracy Skill Instrument (NSI) The Numeracy Skill instrument comprises five distinct components: operation and calculation, graphical representation, quantitative reasoning and logical thinking, complex number (advanced concept), and spatial visualization and spatial reasoning. The Numeracy Skill instrument includes 27 items distributed among the five components, as Table 1 illustrates. Table 1 Initial Component of Numeracy Skill Instrument (NSI) Component Number of Items Operation and Calculation Graphical Representation Quantitative Reasoning and Logical Thinking Complex Number (Advanced Concept) Spatial Visualization and Spatial Reasoning Total 7 4 6 5 5 27 Digital Skills Instrument (DSI) The Digital Skills Instrument (DSI) comprises four components: information literacy, computer and technology literacy, visual literacy, and digital communication/collaboration skills. The instrument encompasses twenty-two items, categorically distributed among the four components. Table 2 presents a detailed breakdown of the number of items assigned to each component. 24 Dalim et al RESSAT 2023, 8(3): 16-35 Table 2 Initial Component of Digital Skills Instrument (DSI) Component Number of Items Information Literacy Computer and Technology Literacy Visual Literacy Digital Communication/Collaboration Skill Total 11 4 3 4 22 Expert Judgment Content Validation A validation process was conducted to validate the Digital Skills Instrument (DSI) and Numeracy Skills Instrument (NSI) through expert evaluation. The process involved obtaining opinions and feedback from a panel of experts specialising in measurement and evaluation. Initially, two experts from both a comprehensive and research universities were selected to evaluate the instruments. Their feedback was then supplemented by an additional nine experts. The experts' evaluations were instrumental in refining the components of the DSI and NSI. Table 3 portrays the experts' expertise, research, and teaching experience in the evaluation. Table 3 Judge and field of expertise Expert Field of expertise Experience (years) 1 2 3 4 5 6 7 8 9 10 11 Language testing (social sciences) Measurement and evaluation (social sciences) Digital literacy (science and technology) Artificial intelligence (science and technology) Instructional communication, new media (social sciences) Psychometric assessment (social sciences) Pharmacy (health science) Statistics and Operational Research Law, criminal justice (social sciences) Electrical Engineering Mathematics Education 18 20 21 15 16 25 20 >25 21 >25 >25 The selection process for volunteer experts followed a rigorous set of criteria, considering various factors such as their background in curriculum development, academic positions they have held, and a minimum of 10 years of research or teaching experience. The experts in the evaluation process represented various disciplines, including science, technology, and social sciences. Their collective expertise and extensive knowledge greatly enriched the evaluation process. It was deemed imperative to measure numeracy and digital skills across a broad spectrum of disciplines and qualification levels, adhering to the standards established by 25 RESSAT 2023, 8(3): 16-35 the MQF. The inclusion of experts from diverse fields enabled the provision of nuanced and informed perspectives on the appropriate numeracy and digital attributes required for various academic programs and curricula within their respective fields. The items were systematically classified according to several criteria Escobar-Pérez and Cuervo-Martínez (2008) recommended, including sufficiency, clarity, coherence, and relevance. Sufficiency ensured that the items within each dimension were comprehensive enough to measure that particular dimension effectively. Clarity ensured that the items were easily understandable. Coherence guaranteed that the items were logically related to the dimension or indicator they were intended to measure. Finally, relevance ensured that the items were considered essential for the evaluation. The experts rated each item on a scale of 1 to 4 and were encouraged to comment in an open space on the instrument. The experts evaluated the questionnaire to determine the dependability of each component and item before pilot-testing the final instrument with potential respondents. A survey was conducted online and disseminated randomly to students from various faculties and disciplines. ANALYSIS and FINDINGS This section presents the results of the data analysis and describes a detailed account of the study’s findings. Content Validation by Expert Judgment The reliability of the data was assessed using Cronbach's Alpha, a statistical method measuring the internal consistency and agreement among experts concerning the content validity of the numeracy and digital skills framework for sufficiency, clarity, coherence, and relevance. The results revealed that Cronbach's Alpha values for all components and numeracy and digital skills characteristics were above 0.8, considered acceptable. Higher values indicated greater internal consistency, reflecting the expert respondents’ consistency and uniformity of opinions and perceptions. Further analysis was conducted to enhance the questionnaires by incorporating the feedback and suggestions provided by the expert panel. The refinement process involved restructuring the sequence of items, refining the language and phrasing, and incorporating two additional components while eliminating the visual literacy component. Some items were also merged with other components to increase their effectiveness. Consequently, the Digital Skills Instrument (DSI) comprised 28 items representing five components: information literacy, computer and technology literacy, digital communication and collaboration skills, digital identity and well-being, and digital ethics. Meanwhile, the Numeracy Skills Instrument (NSI) included five components: operation and calculation, graphical representation, quantitative reasoning and logical thinking, complex number (advanced concept), and spatial visualization and geometric reasoning. Pilot Study Figure 1.0 depicts the distribution of respondents based on college and faculties. In total, 218 students from various academic programs responded to the online survey. 26 Dalim et al RESSAT 2023, 8(3): 16-35 Distribution of Respondents Figure 1 Distribution of respondents according to college and faculties Note: AC = Accountancy; FSKM = Computer and Mathematical Science; FITA = Film, theatre and animation; EDU= Education, COE=Engineering; IM= Information Management; PH=Pharmacy; Business Management; FSG=Applied Science Figure 1 illustrates the distribution of respondents across faculties. As can be seen, 96 out of 218 (44.04%) respondents were from Accountancy, followed by 58 (26.61%) from Computer and Mathematical Sciences, 28 (12.84%) from Film, Theatre, and Animation, and the remaining respondents were from other faculties. The pilot test data were examined using Cronbach's Alpha analysis. Table 4 presents the results of Cronbach’s Alpha analysis for digital skills (DSI) with the items categorised according to the dimensions. The Cronbach's Alpha values for each dimension ranged from 0.847 and 0.958. Table 5 displays the Cronbach's Alpha analysis results for numeracy skills (NSI) items distributed according to dimensions. Cronbach's Alpha was in the range of 0.916 to 0.964 for each dimension in NSI. These Cronbach's Alpha values (Tables 4 and 5) indicate both outcomes’ strong and persuasive internal consistency. The present results could be retained because Cronbach's alpha values of 0.8 or higher are achieved. Table 4 shows that the DSI had strong internal consistency across all five components, with Cronbach’s alpha values between 0.847 and 0.958. The Cronbach's alpha values in DSI were as follows: information literacy (0.958), digital ethics (0.942), digital communication and collaboration skill (0.919), computer and technology literacy (0.895), and digital identity and well-being (0.847). Consequently, all the items were retained to measure digital skills. 9 6 5 8 2 8 2 2 4 4 3 2 1 AC FSKM FITA EDU COE IM PH FBM FSG 27 RESSAT 2023, 8(3): 16-35 Table 4 Item-Total Statistics: Items and Values in Five Components of Perceptions on Digital Skills Instrument (DSI) Component Item Scale Mean if Item Deleted Scale Variance if Item Deleted Corrected Item-Total Correlatio n Cronbach’ s Alpha If Item Deleted Cronbach’ s Alpha Overall C1 – Information Literacy C1.1 110.49 235.726 .687 0.966 0.958 C1.2 110.52 235.264 .738 0.966 C1.3 110.42 235.904 .754 0.966 C1.4 110.60 232.683 .804 0.965 C1.5 110.57 232.689 .794 0.965 C1.6 110.71 234.522 .736 0.966 C1.7 110.76 232.793 .758 0.965 C1.8 110.69 233.006 .790 0.965 C1.9 110.80 231.848 .780 0.965 C1.10 110.59 234.732 .717 0.966 C1.11 110.72 232.590 .749 0.966 C1.12 111.07 235.193 .607 0.967 C2 – Computer and Technology Literacy C2.1 110.51 233.928 .747 0.966 0.895 C2.2 110.99 232.968 .620 0.967 C2.3 110.69 233.790 .741 0.966 C2.4 110.76 233.300 .736 0.966 C2.5 111.13 234.324 .572 0.967 C3 – Digital Communicatio n / Collaboration Skill C3.1 110.69 232.702 .746 .966 0.919 C3.2 110.67 232.016 .718 .966 C3.3 110.89 231.938 .692 .966 C3.4 110.61 234.082 .701 .966 C4 – Digital Identity and Well-Being C4.1 110.31 236.879 .616 .966 0.847 C4.2 110.63 233.745 .654 .966 C4.3 110.40 235.882 .629 .966 C5 – Digital Ethics C5.1 110.39 234.846 .688 .966 0.942 C5.2 110.59 233.691 .714 .966 C5.3 110.44 235.408 .674 .966 C5.4 110.43 235.167 .665 .966 28 Dalim et al RESSAT 2023, 8(3): 16-35 Table 5 Item-Total Statistics: Items and Values in Five Components of Perceptions on Numeracy Skills Instrument (NSI) Component Item Scale Mean if Item Deleted Scale Variance if Item Deleted Corrected Item-Total Correlatio n Cronbach’ s Alpha If Item Deleted Cronbach’ s Alpha Overall B1 – Operation and Calculation B1.1 23.07 1.473 .934 0.920 0.931 B1.2 23.07 1.473 .934 0.917 B1.3 22.96 2.196 .956 0.913 B1.4 23.04 2.238 .334 0.910 B1.5 23.18 1.515 .864 0.913 B1.6 23.18 1.786 .676 0.917 B1.7 23.07 2.307 .214 0.953 B2 – Graphical Representatio n B2.1 11.75 .188 .255 0.933 0.946 B2.2 11.75 .146 .289 0.930 B2.3 11.82 .157 .859 0.928 B2.4 11.82 .057 .859 0.928 B3 – Quantitative Reasoning & Logical Thinking B3.1 17.32 7.286 .397 0.938 0.946 B3.2 17.64 5.122 .630 0.941 B3.3 17.71 5.426 .527 0.936 B3.4 18.04 3.405 .831 0.933 B3.5 17.46 5.217 .765 0.935 B3.6 18.25 4.646 .332 0.935 B4 – Complex Number (Advanced Concept) B4.1 15.36 .622 .624 0.922 0.916 B4.2 15.39 .497 .865 0.893 B4.3 15.43 .536 .752 0.878 B4.4 15.36 .539 .959 0.896 B4.5 15.46 .613 .541 0.890 B5 – Spatial Visualization & Geometric Reasoning B5.1 15.57 .432 .984 0.955 0.964 B5.2 15.57 .432 .984 0.956 B5.3 15.61 .455 .918 0.952 B5.4 15.50 .604 .989 0.956 B5.5 15.46 .696 .642 0.958 Similarly, Table 5 illustrates that the NSI has strong internal consistency across all five components, with Cronbach’s alpha values between 0.916 and 0.964. The Cronbach's alpha 29 RESSAT 2023, 8(3): 16-35 values in NSI were the following: spatial visualization and geometric reasoning (0.964), graphical representation, quantitative reasoning and logical thinking (0.946), operation and calculation (0.931), and complex number (0.916). Therefore, all the items were retained to measure numeracy skills. Furthermore, the corrected item-total correlation indicates that the scales for DSI (0.572 to 0.804) and NSI (0.214 to 0.989) contain highly intercorrelated items. It is plausible when measuring the respondents’ opinions regarding their numeracy and digital skills. Additionally, the absence of corrected inter-item correlation values below 0.214 suggests that none of the scales represent overly broad constructs. These findings suggest that the DSI and NSI are reliable and valid digital and numeracy skills measures. Thus, the instruments can be used confidently in future research. DISCUSSION Our findings led to the development of a framework for numeracy and digital skills, undergoing two phases of repeated validation. During the initial phase, two subject matter experts provided extensive feedback on the content of four constructs: information literacy, computer and technology literacy, visual literacy, and digital communication and collaboration skills. Based on the findings of the first phase of validation, the "visual literacy" component was omitted from the list of components owing to its moderate level of consistency. However, "digital communication and collaboration skills" remained a requirement, because these are the most in-demand talents today. To attain the purpose and benefits of digital technology in the digital world, all learners and consumers must collaborate with others (Van Laar et al., 2017). These characteristics, particularly communication and collaboration abilities, are viewed as essential in the twenty-first century. In the second phase of validation, nine experts participated, and two crucial components were added depending on the relevant items and the experts' feedback. These components addressed the need for digital integrity and the impact of digital technology use on well-being, especially among adolescents (Dienlin & Johannes, 2020). Consequently, the framework includes five components: information literacy, computer and technology literacy, digital communication and collaboration skills, digital identity and well-being, and digital ethics. Figure 2 provides a summary of the digital skill framework. The proposed digital framework is an extension of Belshaw’s (2012) framework, which identified eight key elements necessary for a comprehensive understanding of digital literacy. These elements include understanding the social and cultural contexts of digital technologies, critical and creative thinking, content creation and online community participation, effective communication, positive attitude and digital identity management. Moreover, they have innovative and imaginative use of technology, evaluation and analysis of digital content and sources, and understanding of digital citizenship and ethical online participation. Belshaw has also presented his 5Cs framework for developing digital literacies, comprising critical, creative, confident, collaborative, and civic elements essential for thriving in the digital world. Belshaw 30 Dalim et al RESSAT 2023, 8(3): 16-35 argues that integrating these elements into all aspects of education and training can yield a comprehensive approach to digital literacy. In a more recent study, Bayne and Gallagher (2021) focus on exploring and developing new approaches to digital education. The article highlights the need to prioritise student- centred learning in digital education, including using adaptive learning technologies to tailor learning experiences to individual students' needs and interests. The authors also accentuate the importance of ethical considerations in digital education: issues related to privacy, data security, and online safety. They argue that policies and regulations must be enforced to ensure that digital education is equitable, accessible, and effective. Collaboration and community building are also emphasised as important aspects of digital education. Digital technologies offer new opportunities for learners to connect with each other and with experts in various fields, leading to enhanced learning outcomes. Figure 2 Proposed framework for digital skills INFORMATION LITERACY find relevant, use, organise, elaborate, and manage digital information from different sources, distinguish different types of digital information and its purpose, process data into a more understandable form, reproduce and form new information, decide what to share, explore, develop new ideas/ project or opportunities, and showcase digital or artefact creation) COMPUTER AND TECHNOLOGY LITERACY ability to use and apply computer and software, create graphic design, adapt to learning preferences, and needs, organise, plan, and reflect learning, and knowledge on advanced digital DIGITAL COMMUNICATION / COLLABORATION SKILL convey ideas using multiple digital applications, build networking and collaboration using digital tools/platform DIGITAL IDENTITY AND WELL BEING impact of technology and digital activities, manage online and real-world interaction, the importance of maintaining physical and mental health DIGITAL SKILLS DIGITAL ETHICS acceptable online behavior including how to communicate and treat others, the credibility of the source and proper referencing, compliance with copyright issues, and privacy and security of personal information An individual must first learn how to obtain information and use computers or other digital devices. Therefore, a healthy society emphasises the application of digital technology and knowledge, causing information literacy, computer literacy, and technology literacy, and the 31 RESSAT 2023, 8(3): 16-35 development of new knowledge through communication and collaboration. Additionally, engagement and communication bring everyone closer together, as depicted in the "create" component. However, communication becomes more complicated when multiple parties or individuals are involved. Therefore, using knowledge exploration and social interaction in communication should be monitored for ethical and well-being reasons. Consequently, the "understand" component entails a socialisation process: the responsibilities of individuals in interaction and communication. For the development of digital skills, it is necessary to contemplate a guide on employing digital tools judiciously, including social presence and affective presence in social networks. Moreover, a suitable guide can aid individuals in forming their digital identities (Bozkurt & Tu, 2016). Figure 3 Proposed framework for numeracy skills NUMERACY SKILLS OPERATION AND CALCULATION Basic math calculation, conversion, and answer GRAPHICAL REPRESENTATION Convert numbers into graphs, interpret data in graphical forms, in diagrams, interpret data in tables, solve a wide range of math problems QUANTITATIVE REASONING & LOGICAL THINKING apply fundamental mathematical concepts like algebra, convey mathematical information symbolically, visually, numerically, and verbally, interpret data, draw conclusions, apply mathematical/statistical concepts to predict and draw conclusions SPATIAL VISUALISATION & GEOMETRIC REASONING draw geometric figures in three dimensions, visualise three-dimensional (3D) objects easily, identify three-dimensional (3D) objects in different angles or positions COMPLEX NUMBER use arithmetic methods, algebra, and geometrical methods to solve problems, solve complex numerical routines, and non-routine, using various means and strategies, describe, and compare geometric figures in two dimensions, can identify, describe, and compare geometric figures in three dimensions For the NSI, all five components - spatial visualisation and geometric reasoning, both graphical representation, and quantitative reasoning and logical thinking, operation and calculation, and complex number - are retained, demonstrating that each component is necessary for assessing numeracy skills. Retaining all five components of numeracy skills highlights the multifaceted nature of this construct, with the components interrelated and complementing one another. Mastery in one component can enhance performance in another. For example, strong spatial visualisation and geometric reasoning skills may facilitate more efficient graphical representation and quantitative reasoning (Lowrie et al., 2019). Additionally, 32 Dalim et al RESSAT 2023, 8(3): 16-35 proficient operation and calculation skills may assist individuals in better understanding and applying complex number concepts (National Research Council & Mathematics Learning Study Committee, 2001). Figure 3 illustrates the interconnection of the five components. Thus, it is essential to assess numeracy skills holistically, considering all five components to gain a comprehensive understanding of an individual’s numeracy proficiency. CONCLUSION and RECOMMENDATION We aim to determine both the reliability and validity of the numeracy and digital skills framework using expert opinions and a pilot study. Information literacy, computer and technology literacy, digital communication and collaboration skills, digital identity and well- being, and digital ethics are the five agreed-upon components of digital skills. Before designing courses, learning activities, and assessments during curriculum development, all educators must comprehend the components of digital skills. To generate meaningful digital learning experiences in the classroom, educators must determine their students' digital literacy or ability. The digital skills instrument (DSI) can provide valuable insights into students' digital literacy, serve as a tool for digital requirements analysis, identify students' strengths and weaknesses, and promote the continuous enhancement of curriculum quality. Similarly, the five components of numeracy encompass operation and calculation, quantitative reasoning and logical thinking, graphical representation, spatial visualisation and geometrical reasoning, and complex numbers. On top of the digital skills, these components of numeracy should ensure the quality of current and future curriculum development. Further investigation is needed to enhance the digital skills instrument by incorporating various numeracy-related issues into the existing curriculum. The suggested digital framework may be utilised in education and training programs promoting student-centred learning in digital education to enhance digital skills among learners. By doing so, we can enhance the learning experience and tailor it to each student's requirements and interests. Policies and laws must be in place to guarantee that digital education is egalitarian, accessible, and successful. For the development of digital skills, it is necessary to consider suitable guidance on using digital tools intelligently, including social presence and emotional presence in social networks. Moreover, a suitable guide may aid in the development of digital identities. To get a thorough knowledge of an individual's numeracy competence, it is necessary to test numeracy abilities holistically, taking into account all five components. Future studies might examine the feasibility of implementing the suggested digital skills framework and student- centered learning in digital education. Additionally, research may be conducted to provide a comprehensive guide on how to utilise digital tools effectively and to determine the relationship between the five components of numeracy skills. 33 RESSAT 2023, 8(3): 16-35 Acknowledgement The researchers would like to express their appreciation to all experts who dedicated their time and expertise during the preparation of the survey in the study. This research was supported by a special research grant from Universiti Teknologi MARA (600-RMC/GPK 5/3 (007/2020) and gained ethical approval from the UITM research ethic committee (REC/08/202) MR/713. Availability of data and materials The data supporting the findings of this study are available from the corresponding author upon reasonable request. REFERENCES Aris, S.R.S., Teoh, S.H., Deni, S.M., Nadzri, F.A., & Dalim, S.F. (2022). Digital skills framework in higher education. Proceedings, 82, 61. https://doi.org/10.3390/proceedings2022082061 Bayne, S., & Gallagher, M. (2021). Near Future Teaching: Practice, policy and digital education futures. Policy Futures in Education, 19(5), 607–625. https://doi.org/10.1177/14782103211026446 Belshaw, D. (2012). What is ‘‘digital literacy?’’ A Pragmatic investigation [Doctoral thesis, Durham University]. https://etheses.dur.ac.uk/3959/ Bozkurt, A. and Tu, C.H., (2016). Digital identity formation: Socially being real and present on digital networks. Educational Media International, 53(3), 153-167. https://doi.org/10.1080/09523987.2016.1236885 Bredberg, J. (2020). The role of mathematics and thinking for democracy in the digital society. Policy Futures in Education, 18(4), 517–530. https://doi.org/10.1177/1478210319899242 Furness, J., Cowie, B., & Cooper, B. (2017). Scoping the meaning of ‘‘critical’’ in mathematical thinking for initial teacher education. Policy Futures in Education, 15(6), 713–728. https://doi.org/10.1177/1478210317719778 Dewi, R.S., Hasanah, U. and Zuhri, M., (2021). Analysis study of factors affecting students' digital literacy competency. Ilkogretim Online, 20(3). https://doi.org/10.17051/ilkonline.2021.03.42 Dienlin, T., & Johannes, N. (2020). The impact of digital technology use on adolescent well- being. Dialogues in Clinical Neuroscience, 22(2), 135–142. https://doi.org/10.31887/DCNS.2020.22.2/tdienlin Escobar-Pérez J., Cuervo-Martínez A. (2008). Validez de contenido y juicio de expertos: Una aproximación a su utilización. Av. Med. 6(1), 27–36. https://dialnet.unirioja.es/servlet/articulo?codigo=2981181 European Commission (2015). Digital competence in Europe: evolution and current status. https://ec.europa.eu/jrc/en/publication/digital-competence-europe- evolution-and- current-status https://doi.org/10.3390/proceedings2022082061 https://etheses.dur.ac.uk/3959/ https://doi.org/10.1080/09523987.2016.1236885 https://doi.org/10.1177/1478210319899242 34 Dalim et al RESSAT 2023, 8(3): 16-35 European Commission (2008). The European qualifications framework for lifelong learning. https://ec.europa.eu/ploteus/en/content/european-qualifications-framework-lifelong- learning Ismail, A. A., & Hassan, R. (2019). Technical competencies in digital technology towards Industrial Revolution 4.0. Journal of Technical Education and Training, 11(3). https://publisher.uthm.edu.my/ojs/index.php/JTET/article/view/3208 Kars-Tietema, M., & Kars, E. L. (2018). Preparing students for the future: The importance of numeracy skills in the 21st century. Journal of Education and Practice, 9(19), 128-133. https://doi.org/10.1080/00228958.2011.10516575 Kee, D.M.H.; Anwar, A.; Gwee, S.L.; Ijaz, M.F. (2023). Impact of acquisition of digital skills on perceived employability of youth: Mediating role of course quality. Information 2023, 14, 42. https://doi.org/10.3390/info14010042 Kim, Y., Lee, J., & Lee, K. (2016). The role of numeracy in the prediction of job performance: A meta-analytic review. Journal of Educational Psychology, 108(2), 200–214. https://doi.org/10.1037/edu0000091 Liu, Y. (2016). Digital literacy for lifelong learning in the 21st century. Journal of Educational Technology Development and Exchange, 9, 1-18. https://doi.org/10.3991/jete.v9i0.5685 Malaysian Qualifications Framework (MQF) 2nd edition, (2018). Kuala Lumpur: Malaysian Qualification Agency. https://www.mqa.gov.my/pv4/mqf.cfmMinistry of Higher Education, Malaysia. The Malaysian qualifications framework (MQF). https://www.mohe.gov.my/en/mqf National Research Council & Mathematics Learning Study Committee. (2001). Adding it up: Helping children learn mathematics. National Academies Press. OECD (2013). The survey of adult skills: reader's companion. https://www.oecd.org/skills/The- Survey-of-Adult-Skills-Reader%27s-companion.pdf OECD. (2019). PISA 2018 results (Volume I): What students know and can do. https://doi.org/10.1787/5f07c754-en Rahmawati, S., Abdullah, A. G., Widiaty, I., & Islami, A. R. (2022). The distributions of vocational high school teachers’ advanced digital competence (ADC). Journal of Technical Education and Training, 14(2), 190–201. https://publisher.uthm.edu.my/ojs/index.php/JTET/article/view/10936 Sivarajah, U., Kamal, M.M., Irani, Z. & Weerakkody, V. (2017). Critical analysis of big data challenges and analytical methods. Journal Of Business Research, 70, 263-286. https://doi.org/10.1016/j.jbusres.2016.08.001 UNESCO (2018). A global framework of reference on digital literacy skills for indicators. Information Paper No. 51. Montreal, Canada: UNESCO Institute for Statistics. http://uis.unesco.org/sites/default/files/documents/ip51-global-framework-reference- digital-literacy-skills-2018-en.pdf https://ec.europa.eu/ploteus/en/content/european-qualifications-framework-lifelong-learning https://ec.europa.eu/ploteus/en/content/european-qualifications-framework-lifelong-learning https://ec.europa.eu/ploteus/en/content/european-qualifications-framework-lifelong-learning http://dx.doi.org/10.1080/00228958.2011.10516575 https://doi.org/10.1037/edu0000091 https://doi.org/10.1037/edu0000091 https://doi.org/10.1037/edu0000091 https://doi.org/10.3991/jete.v9i0.5685 https://doi.org/10.3991/jete.v9i0.5685 https://doi.org/10.3991/jete.v9i0.5685 https://www.mohe.gov.my/en/mqf https://www.oecd.org/skills/The-Survey-of-Adult-Skills-Reader%27s-companion.pdf https://www.oecd.org/skills/The-Survey-of-Adult-Skills-Reader%27s-companion.pdf https://doi.org/10.1787/5f07c754-en http://dx.doi.org/10.1016/j.jbusres.2016.08.001 http://uis.unesco.org/sites/default/files/documents/ip51-global-framework-reference-digital-literacy-skills-2018-en.pdf http://uis.unesco.org/sites/default/files/documents/ip51-global-framework-reference-digital-literacy-skills-2018-en.pdf 35 RESSAT 2023, 8(3): 16-35 Van Laar, E., Van Deursen, A.J., Van Dijk, J.A. and De Haan, J., (2017). The relation between 21st-century skills and digital skills: A systematic literature review. Computers In Human Behavior, 72, 577-588. https://doi.org/10.1016/j.chb.2017.03.010 Lowrie, T., Logan, T., & Hegarty, M. (2019). The Influence of Spatial Visualization Training on Students’ Spatial Reasoning and Mathematics Performance. Journal of Cognition and Development, 20(5), 729–751. https://doi.org/10.1080/15248372.2019.1653298 Vodă, A.I., Cautisanu, C., Grădinaru, C., Tănăsescu, C. and de Moraes, G.H.S.M., (2022). Exploring digital literacy skills in social sciences and humanities students. Sustainability, 14(5), 2483. https://doi.org/10.3390/su14052483 Vrana, R. (2016). Digital Literacy as a Boost Factor in Employability of Students (pp. 169–178). https://doi.org/10.1007/978-3-319-52162-6_17 World Bank (2015). Digital dividends: world development report 2016. Retrieved from http://www.worldbank.org/en/publication/wdr2016 Yamashita, T., Punksungka, W., Narine, D., Helsinger, A., Kramer, J. W., Cummins, P. A., & Karam, R. (2023). Adult numeracy skill practice by STEM and non-STEM workers in the USA: an exploration of data using latent class analysis. International Journal of Lifelong Education, 42(1), 59–76. https://doi.org/10.1080/02601370.2022.2146772 https://doi.org/10.1016/j.chb.2017.03.010 https://doi.org/10.3390/su14052483 http://www.worldbank.org/en/publication/wdr2016