Indian Journal of Educational Technology Vol. 2 (1), January 2020 64 Research Article Interactive Effect of Meta-Cognitive Strategies-based Instruction in Mathematics and User experience on Student-Outcomes Meenakshi Ingole Assistant Professor, Department of Education, University of Delhi, India, Email: meenakshi.ingole21@gmail.com) Abstract This paper attempts to ascertain the interactive effect of meta-cognitive strategies based instruction in mathematics and learner (user) experience design. This study focuses more on meta-cognitive experiences than meta-cognitive skills. For this purpose, an intervention programme based on meta-cognitive strategies of about 35 hours was developed for students of standard eighth spreading over eight weeks. The aim of the research was to ascertain whether meta-cognitive strategies-based instruction facilitates the academic achievement of students and user experience and interaction design decisions significantly affect how well on users (students) learn through meta-cognitive strategies. Structured tools were used in study. The participants of the study included 62 and 60 students in the experimental and control groups respectively. Students were found to be significantly influenced by the intervention programme as well as to help user interface in a way that supports and enhances the cognitive domain. The effect size of the intervention programme on academic achievement of students was found to be 1.30 and mathematics anxiety 1.084 was high in magnitude. Keywords: Meta-cognitive strategies, User experience, Academic achievement, Mathematics anxiety Introduction According to George Polya, we can think of two kinds of aims for school education: a good and narrow aim that of turning out employable adults who (eventually) contribute to social and economic development; and a higher aim, that of developing the inner resources of the growing child. With regard to school mathematics, the former aim specifically relates to numeracy. It affects the usability, simplicity and clarity of content. It also affects the way users conceive of interactive possibilities. Since usability is known to be an important factor in how deeply, how easily, and how successfully a user moves through the content of an environment, the more usable learning environment is the more successful it will likely be. According to Indian National curriculum framework: Problems in teaching and learning of mathematics Following are the four problems which have been deemed to be the core areas Indian Journal of Educational Technology Vol. 2 (1), January 2020 65 of concern: 1. A sense of fear & failure regarding mathematics among a majority of children 2. A curriculum that disappoints both a talented minority as well as the non-participating majority at the same time 3. Crude methods of assessment that encourage perception of mathematics as mechanical computation 4. Lack of teacher preparation and support in the teaching of mathematics. These problems encourage us to think about how we enhance students’ academic achievement and reduce anxiety towards mathematics. There is a reason, rarely if ever mentioned, why good visual design can facilitate learning. It can improve meta- cognition. That is my main objective here. It is not standard to associate visual design with meta-cognition. Meta- cognition, in its most basic form, is the activity of thinking about thinking. Since thinking is often taken to be a mental activity, largely a matter of manipulating internal representations, there has been little reason to look to the structure of the environment as a factor in thinking. According to Alben (1996) All the aspects of how people use an interactive product: the way it feels in their hands, how well they understand how it works, how they feel about it while they’re using it, how well it serves their purposes, and how well it fits into the entire context in which they are using it. In principle, UXD (user experience design) is not different from HCD (Human centre design). However, UXD adds important dimensions to the challenge of implementing HCD in a mature form. These additions are not trivial. The main dimensions distinguishing UXD from a traditional view of HCD include UX factors; methods, tools and criteria used in UX work; representation of the UX idea; and UX positioning in the organization. UX matters, Encompasses all aspects of a digital product that users experience directly—and perceive, learn, and use—including its form, behaviour, and content. Learn ability, usability, usefulness, and aesthetic appeal are the key factors in users’ experience of a product. According to McNamara & Kirakowski (2006), the user experience considers the wider relationship between the product and the user in order to investigate the individual’s personal experience of using it. What are User experiences? User experiences are the methods that have the objective of improving user satisfaction with achieving both pragmatic and hedonic goals. UX is a consequence of a user’s internal state (predispositions, expectations, needs, motivation, mood, etc.), the characteristics of the designed system (e.g. complexity, purpose, usability, functionality, etc.) and the context (or the environment) within which the interaction occurs (e.g. organizational /social setting, meaningfulness of the activity, voluntariness of use, etc.). Obviously, this creates innumerable design and experience opportunities. Indian Journal of Educational Technology Vol. 2 (1), January 2020 66 Review of literature According to Kirsh (1995), the way visual cues are structured and the way interaction is designed can make an important difference in the ease and effectiveness of cognition and meta- cognition. Documents that make effective use of markers such as headings, callouts, italics can improve students’ ability to comprehend documents and ‘plan’ the way they review and process content. Interaction can be designed to improve ‘the proximal zone of planning’ – the look ahead and apprehension of what is nearby in activity space that facilitates decisions. This final concept is elaborated in a discussion of how e-newspapers combine effective visual and interactive design to enhance user control over their reading experience. (Howe at all, 2000); Lucangeli, Coi and Bosco (1997) found that fifth graders viewed problems containing large numbers as more difficult than problems with smaller numbers in their study examining the meta-cognition of mathematics difficulty in elementary school children. In this study, students who were classified as poor problem solvers showed lower meta-cognitive awareness and made more errors when solving problem. Such learners keep planning in mind during the learning activity and assess their performance against them. During the learning activity, teachers can encourage learners to share their progress, their cognitive procedures and their views of their conduct. The emphasis is on the affective character of ME, which has received little attention in the past, Efklides (2006). Unlike online task-specific knowledge, which is conscious and analytic, the other ME are products of non- conscious, non-analytic inferential processes. Because of their nature, ME can trigger either rapid, non-conscious control decisions or conscious analytic ones. However, ME can make use of both the affective and the cognitive regulatory loops, and this has a series of implications for learning. Evidence is presented regarding the relations of ME with affect and cognition, and the implications of the lack of accuracy of ME for the self-regulation of learning. Particular emphasis is given on judgment of learning, feeling of difficulty, and feeling of confidence. The challenges for future research on meta-cognition are underscored. Need of the Study Prior research, especially on Indian students from lower socio-economic background and with an average ability has not been conducted with a view to enhance such students’ knowledge about cognition and knowledge about regulation. Meta-cognitive awareness forms a cognitive doctrine and meta-cognition could begin when cognition fails. It is essential to study the interactive influence of meta-cognitive strategies- based instruction and user experience of student outcomes in mathematics. Meta-cognition enables students to benefit from instruction (Carr, Kurtz, Schneider, Turner & Borkowski, 1989; Van Zile-Tamsen, 1996) and influences the use and maintenance of cognitive strategies. In the present research, there is an Indian Journal of Educational Technology Vol. 2 (1), January 2020 67 attempt to study the effect of meta- cognitive strategies-based instruction in mathematics on students’ levels of Knowledge about Cognition (procedural knowledge, declarative knowledge and conditional) in terms of User experience on student academic achievement and mathematics anxiety. Operational Definition of the Terms • Meta-Cognition:Meta-cognition refers to learner’s awareness of their own knowledge & cognitive processes & their ability to understand control and manipulate their own cognitive processes. • User Experience: Every aspect of the user’s interaction with a product, service, or company that makeup the user’s perceptions of the whole. UE works to coordinate these elements to allow for the best possible interaction by users. • Meta Cognitive Strategies :Meta- cognitive strategies refers to methods used to help students understand the way they learn and refers to the processes designed for students to manage, monitor and evaluate their learning and ‘think’ about their ‘thinking’. Statement of the Problem Interactive Effect of Meta-Cognitive Strategies-based Instruction in Mathematics and User experience on Student-Outcomes Scope and Delimitations of the Study In the present study, English medium schools from the Greater Mumbai affiliated to the SSC board have been included. It excludes schools with other media of instruction such as Marathi, Hindi, Urdu, Gujarati etc. The present study includes eighth standard from English medium schools situated in Greater Mumbai. Students from other primary and secondary classes have been excluded. It also excludes schools affiliated to ICSE or CBSE boards. Aim of the Study The aim of the study was to ascertain the interactive effect of the intervention programme as user experience on academic achievement & mathematics anxiety of student. Research Questions 1. Do Meta-cognitive strategies & user experiences effect on academic achievement of eight standard students? 2. Do Meta-cognitive strategies & user experiences effect on mathematic anxiety of eight standard students? 3. What is the effect size of the intervention programme (UX) on the academic achievement and mathematics anxiety in the subject of Mathematics? Methodology of the Study Mixed method approach has been used for collecting a data, Qualitative data has been collected through classroom observations and quantitative data has recorded through experimental research which is called as, embedded design. According to Pandya (2011), the study has adopted the quasi- Indian Journal of Educational Technology Vol. 2 (1), January 2020 68 experimental method. In the present research, the quasi-experimental design of the pre-test post-test, non-equivalent group type was used. It can be described as follows: The pre-test-post-test non-equivalent groups design: O1X O2 O3 C O4 Where, O1 and O3: Pre-test Scores & O2 and O4: Post- test Scores X: Experimental Group & C: Control Group Tool of the Study: In the present study, following tools were used by the researcher to collect data quantitatively: 1. Mathematics Anxiety Rating Scale (MARS-I) (Karimi, 2008) 2. Academic Achievement Test (Ingole, 2013) Prototype/Intervention Programme The duration of the intervention programme is 35 hours. The control group was taught using the traditional method. The experimental group was taught using intervention programme, which was divided into two levels. The first level included knowledge about cognition, which was ascertained through achievement test and KWL chart. The second level included regulation about cognition which consisted of three steps, namely, planning (understanding the problem, devising a plan, carrying out the plan & looking back), monitoring (self - awareness of one’s thought processes), control (self-monitoring of one’s thought processes, beliefs and intuitions about one’s cognition) and evaluation (problems on the topic and self -reflection sheet).The three step process is explained further using the following questions: (a) Planning: What is the nature of the task? What is my goal? What kind of information and strategies do I need? How much time and resources do I need? In term of user experience this is IDEAT (b) Monitoring: Do I have a clear understanding of what I am doing? Does the task make sense to me? Am I reaching my goals? Do I need to make changes? In terms of user experience, this is PROTOTYPE (c) Evaluating: Have I reached my goal? What worked? What didn’t work? Would I do things differently the next time? The meta- cognitive strategies included in the study were (a) Knowledge about cognition, (b) Regulation about cognition, (c) Ask questions, (d) Foster Self-reflection, (e) Encourage self- questioning, (f) Think aloud & (g) Self- explanation. The teaching units were selected from the syllabus prescribed for the schools affiliated to the SSC board for the state of Maharashtra and included the topics on Cube, Indices, Construction of Quadrilateral, Joint Bar Graph and Discount and Commission. In term of user experience this is TEST/ Evaluate. While working with meta-cognitive strategies in form of user experience or interactive product are given below. 1. Initiate – What are challenge faced by students while learning Mathematics? 2. Investigate – Personas will be eight standard students & will investigate actual problems of mathematic anxiety. 3. Inspire & ideate – Do a benchmarking analysis & define the user scenarios Indian Journal of Educational Technology Vol. 2 (1), January 2020 69 (benchmarking is inform of instructional design in this case meta- cognitive strategies where ideate for making mathematics content more fragmented and simpler) 4. Prototype - Lesson plan are based on meta-cognitive strategies like Think aloud, KWL charts, concept map, Graphic organizers & self-reflection sheets. 5. Evaluate and reflect – For evaluation Reflection of think aloud sheet, self-reflection sheet were used. Initially, researcher focus on the selection of the right resource and the actions to be carried out are decided together with the faculty members interested in the project. Activities have been conducted for 8th standard enact class. Techniques of Data Analysis: The present research used statistical techniques of ANCOVA and Wolf’s formula. a) Data Analysis: Null Hypothesis 1: There is no significant difference in the pre-test scores of students from the experimental and control group on the following variables: a) Academic Achievement b) Mathematics Anxiety This hypothesis was tested with the objective of comparing the initial status of the experimental and control groups on dependent variable viz. academic achievement and mathematics anxiety of students. The technique used for testing the null hypotheses is the t-test. Table-1 shows the relevant statistics of the scores of academic achievement and mathematics anxiety of students of control and experimental group Table - 1: Relevant Statistics of Pre-Test on The Dependent Variables of CG and EG No. Variable Group N Mean SD T P (two-tailed) LoS 1. AAS CG 60 5.316 2.266 0.03 0.98 N.S EG 62 5.306 2.236 3. MAnS CG 60 79.23 19.881 0.41 0.682 N.S EG 62 80.62 19.881 Null Hypothesis 2: There is no significant difference in the post- test scores of students of experimental group and control group of following variables. a) Academic Achievement b) Mathematics Anxiety Table - 2 shows the post-test AAS of EG and CG after partialling out effect of pre-test AAS. Table -2: Post –Test AAS of EG and CG Groups Observed Mean Adjusted Mean Experimental 14.61 14.61 Control 9.61 9.61 Indian Journal of Educational Technology Vol. 2 (1), January 2020 70 Table-3: ANCOVA for Post- Test Mean of AAS of EG and CG Sources of Variation SS df MS F-ratio P Adjusted A 762.45 1 762.45 96.14 <0.0001Adjusted B 6.51 2 3.25 Adjusted A×B 92.85 2 In Table-3 it can be seen that the ‘P’ value is <.0001, Hence a significance difference is found between the EG and CG. Thus, the null hypothesis is rejected. It can be stated that there is a significant difference in the post-test scores of students’ academic achievement of experimental group and control group. Mean AAS of students from EG is significantly greater than that of CG. Table - 4: Post –Test MAnS of EG and CG Group Observed Mean Adjusted Mean Experimental 49.56 49.34 Control 61.76 61.99 ANCOVA determines whether these two adjusted post-test means differ significantly from each other. Table-5 shows the relevant statistics of ANCOVA for post-test mean MAnS of EG and CG. Table - 5: ANCOVA for Post- Test Mean of MAnS of EG and CG Sources of Variation SS df MS F-ratio P Adjusted A 4874.17 1 4874.17 36.97 <.0001Adjusted B 96.4 2 48.2 Adjusted A×B 1268.16 2 Table - 6: Effect Size of the Treatment and LA on the Dependent Variables Dependent variables Effect Size Academic Achievement 1.30 Mathematics Anxiety 1.084 Qualitative Data Analysis: Observation was tool used for collecting qualitative data, as approach was embedded design, during a quantitative experiment, the researcher may collect qualitative data to examine how participants in the treatment condition are experiencing the intervention, behavioural and attitudinal metrics of usability (i.e. users’ performance and satisfaction). Indeed, different attempts Indian Journal of Educational Technology Vol. 2 (1), January 2020 71 have been undertaken to demarcate or even dismiss the boundary between usability and UX at the conceptual as well as operational level (e.g. Law et al., 2008). Mainly two themes that come from observation are: first, usability is subsumed by UX; second, UX is an elaborated form of satisfaction, which was checked through self-reflection sheets and Think aloud sheets in Mathematics. Conclusion Implications for the learning outcomes Students of the experimental group are found to have a higher academic achievement and reduced mathematics anxiety as compared to those from the control group. In other words, the meta-cognitive strategies-based intervention programmed in Mathematics has been effective in enhancing academic achievement and reducing mathematics anxiety among students of standard eighth. Discussion In the experimental group, user (learner) experience about meta- cognitive strategies such as think aloud strategies, KWL charts, concept map, planning and monitoring and self- reflection sheets. These are expected to facilitate awareness of personal success, beliefs for factors that influence academic performance in mathematics, knowledge of ways for effective learning, knowledge of strategies that should be used. This in turn would have enhanced students’ confidence in learning mathematics, thereby enhancing their problem solving ability. This ultimately could have facilitated higher academic achievement and reduced mathematics anxiety in students of the experimental group as compared to those from the control group. Having outlined the affective and cognitive modes of regulation through which UX (learner) effect behaviour, we come now to some intuitive or counterintuitive implications the functioning of UX (learner) has for the learning outcomes. There are two issues that are of importance here: the accuracy of UX (learner) & the effects of the particular UX (learner) on learning. The response of UX (learner) is a very important issue because it has a bearing on the efficiency of the control decisions in learning situations with respect to effort allocation, time investment, or strategy use. There are various possible reasons why a meta- cognitive judgment is not accurate. The first one is that ME are based on non-conscious, heuristic, inferential processes that make use of various cues, which regard the task and its presentation or the fluency of processing (Koriat, 1997; see also Whittlesea, 1993). This user experience does not only include usability, but also other cognitive, socio-cognitive and affective aspects of users’ experience in their interaction with artefacts(meta- cognitive strategies), such as users’ enjoyment, aesthetic experience, desire to repeat use, positive decision to use a digital artefact and enhanced meta- cognitive experiences. Furthermore, finding was user Indian Journal of Educational Technology Vol. 2 (1), January 2020 72 experience (UX) manifests as quality in design, in interaction and in value, with diverse measures from many methods and instruments. While observation of class researcher faced challenges related to UX is how to select appropriate measures to address the particularities of an evaluation context. According to Effie (2010), the necessity and utility of UX measures is apparent, because such measures enable professionals to benchmark competitive design artefacts & to select appropriate design options. However, both the construct validity and predictive power of some UX measures are of particular concern. Consequently, modelling users’ experience – as a basis for producing design guidance – is especially important. References Chick, N. Meta-cognition, centre for teaching, Venderbelt University, retrieved from blog https://cft.vanderbilt.edu/guides-sub-pages/metacognition/ Denzin, N. K. and Lincoln, Y. S. (Edt) (1994) Handbook of Qualitative Research. Thousand Oaks, CA: Sage Efklides, A. (2006). Meta-cognition and affect: What can meta-cognitive experiences tell us about the learning process? Educational research review 1(1) 3-14Retrieved from http://www.sciencedirect.com/science/article/pii/S1747938X06000029 Effie, L, Low, C., Paul van Schaik (2010) Modelling user experience – An agenda for research and practice. Interacting with Computers, 22, (5), 313–322 retrieved from https:// academic.oup.com/iwc/article/22/5/313/682089 Law, E.L.-C., Nguyen-Ngoc, A.-V.(2008). Fostering self-directed learning with social software: social network analysis and content analysis. In: Proceedings of European Conference of Technology-Enhanced Learning (EC-TEL’08), Maastricht, The Netherlands, 203–215 Kirsh, D. (1995). “The Intelligent Use of Space”. Artificial Intelligence, 73, (1-2), 31-68 Kirsh, D. (2004). Meta-cognition, Distributed Cognition and Visual Design, cognition, education and communication technology, retrieved from http://adrenaline.ucsd.edu/ Kirsh/Articles/Metacognition/metacognition.pdf Marchitto, M. and Cañas, J. (2011). User Experience as a challenge for cognitive psychology and ergonomics Human technology An Interdisciplinary Journal on Humans in ICT Environments, 7(3), 268–280 McNamara & Kirakowski. (2006). Retrieved from http://www.allaboutux.org/ux-definitions Pandya, S. (2011). Educational Research: New Delhi, Himalaya publication. Polya, G. (1969.). “The goals of Mathematical education”, in Communicator, the magazine of the California Mathematics Council Weimer, Maryellen. (2012). Deep learning vs. surface learning: Getting students to understand the difference. Retrieved from the Teaching Professor Blog from http:// www.facultyfocus.com/articles/teaching-professor-blog/deep-learning-vs-surface- learning-getting-students-to-understand-the-difference/. UXPA, Usability Body of Knowledge, Glossary. retrieved from https://uxpa.org/resources/ definitions-user-experience-and-usability