Pa ge 1 Pa ge 1 American Journal of Education and Technology (AJET) The Effect of Chatgpt 3.0 as a Personalized Learning Tool in Answering Atomic Structure Assessments Nick Andrei V. Cayetano1*, Francisza C. De Dios1, Leonardo M. Francisco2, Ishbel Zanthie N. Hernandez1, Frances Lei V. Sequito1, Sophia Marie C. Sevilla1, Louise Beatriz Sison1 Volume 4 Issue 3, Year 2025 ISSN: 2832-9481 (Online) DOI: https://doi.org/10.54536/ajet.v4i3.4700 https://journals.e-palli.com/home/index.php/ajet Article Information ABSTRACT Received: April 30, 2025 Accepted: May 03, 2025 Published: June 17, 2025 This study investigates the effect of ChatGPT 3.0, on Grade 11 STEM students’ performance in Atomic Structure assessments. Using a one-group pretest-post-test quasi-experimental design, the study applied the Wilcoxon Signed Rank test, Cohen’s D, and Hake’s Gain index to measure changes in cognitive assessment scores and technology acceptance via a TAM survey. Results revealed a significant increase in test scores with a large effect size (Cohen’s D = 0.9), though gains in specific subtopics were modest. The findings suggest that ChatGPT 3.0 can enhance learning outcomes and promote AI acceptance in education, warranting further investigation with extended intervention periods and larger sample sizes.Keywords Atomic Structure, ChatGPT 3.0, General Chemistry Assessment, Generative AI, TAM Survey 1 St. James Academy, Malabon City, Philippines 2 De La Salle University, Manila, Philippines * Corresponding author’s e-mail: nick.cayetano018@gmail.com INTRODUCTION Artificial Intelligence (AI) goes back to the 1950s when John McCarthy organized a two-month workshop at Dartmouth College, USA. In the workshop proposal, McCarthy used the term AI for the first time in 1956 (Russell & Norvig, 2010). The premise behind the study of AI is that every facet of learning or any other intelligence component can theoretically be characterized with such precision that a computer may be created to mimic it. Their set goal is to figure out how to make machines speak, create abstractions and notions, solve issues that are currently only solved by humans, and develop by themselves. Baker and Smith (2019) provided a broader definition of AI, “computers which perform cognitive tasks, usually associated with human minds, particularly learning and problem-solving”. Its innovation has significantly impacted the society in which humans live today. The advent of Generative Adversarial Networks (GANs) following AI creation became a more profound and novel model. A GAN consists of two neural links: the generator and the discriminator. This neural network has the potential to make education more interactive and inclusive for students. Unesco (2023) noted that the ability of Generative AI (GAI) to align with learning, interest, pace, ability, and style enables it to provide immediate feedback. AI serves as a catch-all phrase for a variety of tools and techniques, including algorithms, machine learning (ML), natural language processing, data mining, and neural networks. AI and ML are often mentioned in the same context. ML is a method of AI for supervised and unsupervised classification and profiling, for example, to predict the likelihood of a student dropping out from a course or being admitted to a program, or to identify topics in written assignments. Popenici and Kerr (2017) defined ML “as a subfield of AI that includes software able to recognize patterns, make predictions, and apply newly discovered patterns to situations that were not included or covered by their initial design”. Rahman et al. (2021) used ML for cow price prediction and achieved a 70% accuracy through pattern recognition. A popular example of ML is the Chat Generative Pre- trained Transformer (ChatGPT) — a sibling model to InstructGPT that is programmed to follow a prompt or instruction given by a user to provide a detailed response. Kasneci et al. (2023) stated that ChatGPT is backed by OpenAI, which uses a large language model (LLM). The model acknowledges personalized learning experiences and educational content. Improving student engagement when utilized, which overall contributes to effective education delivery. In addition, Susarla et al. (2023) noted that ChatGPT can assist with data collection and problem formulation. Two of these studies acknowledge the potential of ChatGPT as a modern education tool that can be utilized in the school setting. ChatGPT is fine-tuned to produce responses that are similar to humans in a conversational tone giving its users the ability to receive information by typing a prompt (input). According to Dasborough (2023), the ability of ChatGPT to create user content is powered by GAI, which supplies texts and creative contents from different resources. Similar to the study by Qiao et al. (2022), the model ChatGPT can be interrelated via text-to- image generative models. Technology used in AI continues to upgrade its quality, as observed in the performance gap between GPT-2 and the more recent GPT-3. Pa ge 2 https://journals.e-palli.com/home/index.php/ajet Am. J. Educ. Technol. 4(3) 1-11, 2025 GPT-2 began as an AI used for document summarization as it analyzes given/prewritten text, while GPT-3 can perform more complicated tasks such as answering questions, language translation, conducting advanced searches, and all of the capabilities of GPT-2. GPT-3 has been trained on more textual data and generally provides better responses compared to its predecessor GPT-2. According to Fullestop (2024), GAI have impacted the learning of students in many different ways because of the huge role it plays in the education system. The ability of GAI to understand the strengths and weaknesses of students allows it to supply assistance and tutoring when it comes to learning. Tamer and Knidri (2023) stresses the use of AI, chatbots, and modern technologies to better respond to the needs of the current age of knowledge and globalization. ChatGPT offers personalized learning style and materials to understand a lesson. A philosophical question remains whether these machines will be able to actually think or even develop consciousness in the future rather than just simulating thinking and showing rational behavior. GAI’s integration as a learning tool has become prominent in recent years. Researchers had seen huge potential of ChatGPT in the education system. Its extent remains underexplored. Shifting the focus on testing its integration in specific content areas in the Atomic Structure topic under the General Chemistry assessment course. The usage of GAI has become essential in education. Yet, its influence towards its acceptance has not been thoroughly examined. In order to address these knowledge gaps, the researchers had chosen to explore these three research questions (RQs) such as: 1. Does Generative AI affect the participants assessment score related to their pretest score after the intervention? 2. How does Generative Al affect the participants performance in the Basic Subatomic Particles (BSP), Atomic Model (AM), and Atomic Composition (AC) subpart of the assessment after the intervention? 3. How are the participants Perceived Ease of Use (PEU), Perceived Usefulness (PU), and Behavioral Intention (BI) influenced by Generative AI after the intervention? LITERATURE REVIEW The rise of AI has led to its application across many disciplines, particularly in healthcare and education. AI’s influence in these areas is largely due to its ability to be trained using data. In education, AI is proving beneficial through its more comprehensive applications. For instance, Keddy (2022) emphasized that a growing body of literature suggests the integration of ChatGPT can cultivate creativity and innovative thinking, especially among business students. AI systems, by using distinct prompts according to the desired output, can generate human-like content in the form of texts, images, or videos (Alasadi & Baiz, 2023; Chiu, 2023; Lee et al., 2023; Pavlik, 2023). Generative AI’s ability to provide content instantaneously is rapidly transforming the educational landscape. According to Bolick and da Silva (2024), AI-driven content generation has become an essential part of instructional design, requiring education professionals to be adept at utilizing these tools effectively. GAI tools such as ChatGPT, Gemini, and Dall-E are increasingly being used by professionals in classroom settings, fostering adaptability and encouraging acceptance among students. In addition to generative tools, a significant development in AI technology is the Generative Pre- trained Transformer (ChatGPT). Brown et al. (2020) highlighted the evolution of GPT-3, developed by OpenAI, which enhances task-agnostic AI by using 175 billion parameters. Ayoub (2020) argued that the changes AI brings to education will enhance student success and better prepare them for the professional world. Now that the researchers have given the overview and role of GAI, it is time to review the implication of this technology to the Science, Technology, Engineering, & Mathematics field (STEM Field). ML allows the model to self-learn and change towards analyzing multiple levels of abstraction (Lecun et al., 2015). This innovation has extended its influence beyond education, impacting fields such as engineering, journalism, medicine, economics, and finance. The integration of GAI has led to its use in impact assessments across various sectors. Notably, the Ada Lovelace Institute (2022), conducted an assessment within the United Kingdom’s National Health Service (NHS), focusing on the applications of GAI. However, there are academic concerns about AI in education, particularly related to ethical standards, transparency, and accountability (Kim & Kim, 2022). While these challenges exist, GAI, like ChatGPT, offers significant opportunities for higher education, including instant information retrieval, personalized learning, and data analysis. In the field of Chemistry, the DENDRAL project was the first to adapt the utilization of AI. Appeared first at Stanford University, which developed methods for predicting compounds structured from their spectrum of mass. In addition, AI and ML technology fosters a better understanding of molecular scales. Which contributed to a deeper understanding of science enthusiasts. In fact, the Sumitomo Dainippon Pharmaceutical Co. and Exscientia, which are AI companies, have developed a drug that is in phase 1 clinical trials. An estimated 4.5 years of hard work has been accomplished for a year, saving massive amounts of money. The Increasing interest in education has been stimulated by AI. In an effort to enhance student achievement, AI is being incorporated into the classroom with greater frequency and in a variety of ways, despite its relatively recent initialization (Garcia-Martinez et al., 2023). GAI supplements the students to grasp their course content. Complementing the ability of GAI to keep track of how well students perform over several assessment loads. Studies have shown that engaging students with Pa ge 3 https://journals.e-palli.com/home/index.php/ajet Am. J. Educ. Technol. 4(3) 1-11, 2025 GAI correlated to improvement and an increase in the motivation level by offering immediate and interactive learning experiences that are personalized (Kabudi et al., 2021). Luckin et al. (2016) described three categories of AI software applications in education that are available today: a) personal tutors, b) intelligent support for collaborative learning, and c) intelligent virtual reality. Intelligent Tutoring Systems (ITS) can be used to simulate one-to-one personal tutoring. Based on learner models, algorithms, and neural networks, they can make decisions about the learning path of a student and the content to select, provide cognitive scaffolding, and help to engage a student in dialogue. ITS has enormous potential, especially in large-scale distance teaching institutions, which run modules with thousands of students, where human one- to-one tutoring is impossible. A vast array of research shows that learning is a social exercise; interaction and collaboration are at the heart of the learning process (Jonassen et al., 1995). Artificial Intelligence in Education (AIEd) can contribute to collaborative learning by supporting adaptive group formation based on learner models, facilitating online group interaction, or summarizing discussions that can be used by a human tutor to guide students towards the objective of a course. Finally, also drawing on ITS, Intelligent Virtual Reality (IVR) is used to engage and guide students in authentic virtual reality and game- based learning environments. Virtual agents can act as teachers, facilitators, or student peers in virtual or remote labs (Perez et al., 2017). AI is capable of generating fast evaluation and feedback. For a continuous evaluation of student performance, AIEd can be incorporated into learning activities as an alternative to stop-and-test. ChatGPT can assist students in understanding and solving assessment problems. AI-powered systems can analyze students’ answers and provide tailored hints, explanations, or additional resources to reinforce learning. Such assessments can also track progress over time, identifying knowledge gaps and adjusting the difficulty of tasks to suit each students needs and capabilities. The use of GAI to conjure virtual simulations in the Chemistry field also provides hands-on experience while ensuring the safety of those involved. As GAI is accessible through the internet, students can learn anything, anytime, anywhere from the web. The task of GAI to provide an automated assessment of the students’ learning resources is helpful for them to improve their knowledge in solving Chemistry problems, especially when the technology has the power to identify a student’s learning capabilities (Das et al., 2021). Therefore, students having virtual simulations for Chemistry assessments readily available has a great impact on their learning journey. High-accuracy predictions of a student’s likelihood of failing an assignment or quitting a course have been made using algorithms (Bahadir, 2016). GAI is having a profound impact on education, particularly when it comes to assisting students and teachers with assessments. According to Tsai et al. (2021), the increasing digitalization of educational resources and the popularity of online learning platforms has caused GAI to play a major role in students’ learning ability for a subject and in teachers’ strategies to test questions. GAI can generate a set of questions that are less complicated for students to understand. Along with the generated questions, GAI provides teachers with a set of repetitions of questions and examples that can help students improve their ability to solve problems related to their lessons (Plessis, 2020). When GAI assists teachers in creating questions on complex concepts, it enables students to answer test questions more efficiently and understand the concepts in a deeper manner. In fact, Tsai et al. (2021) emphasized that using GAI to create questions on complex topics for a test improved students’ learning performance and deepened their knowledge about the topic. GAI can create customized testing experiences by generating questions, creating scenarios, or even tasks unique to a user’s responses or progress. Ensuring that assessments are matched according to an individual’s overall performance as well as their learning patterns, considering how this is unique to each person. Personal assessment tools complement this method well by providing insights into a user’s strengths, weaknesses, or cognitive/emotional states. When all of this is combined, these technologies enable a more precise and adaptive testing environment. Optimizing factors like learning pace, knowledge gaps, and personal growth. This approach is beneficial in skill development and psychological assessments. It provides a deeper understanding of a person’s capabilities. Such applications help build confidence and understanding in an increasingly complex and rapidly changing world. The innovative use of ChatGPT presents transformative opportunities for education management. It has the power to reshape traditional teaching methods by adapting AI-driven learning systems. Zhu and Whang (2019) emphasized that intelligent learning environments can adaptively push personalized learning experiences, while also helping students find resources efficiently. Personalized learning is crucial, as learning experiences differ from one individual to another. According to Lim (2023), with GAI rapidly emerging as a transformative innovation along the lines of the internet and the smartphone, there is now a golden opportunity to truly reimagine and transform the future of education. The use of AI is expanding in learning, including natural sciences like chemistry (Ardyansyah, 2024). The 21st century is seeing the unprecedented and reflective resolution of the GAI pace, which industries and new social spaces are drawn out globally. In line with acknowledging its disruptive potential, the Philippines has initiated integrating AI into its educational system but is cautiously and strategically doing so. Acknowledging AI’s transformative potential, it is significant for students to understand the basics of AI even though the amendment itself is not purely Pa ge 4 https://journals.e-palli.com/home/index.php/ajet Am. J. Educ. Technol. 4(3) 1-11, 2025 targeted at AI, an update in the K to 12 curriculum by the Department of Education (DepEd) to infuse some concepts on computational thinking and elementary programming. The Commission on Higher Education (CHED) has also been proactive, calling on universities to roll out specialized courses and programs in data science, ML, and AI so that there will eventually be an appropriately qualified workforce. The infusing of AI in Philippine education has brought many possibilities where the entire learning experience can be enhanced, including technologies that support the creation of new assessment modes and upskilling programs, making quicker and more precise feedback available to learners and teachers. AI will also improve the teachers’ understanding of how students learn and be able to change how they teach accordingly. While much is promised by AI, there are also issues concerning AI applications for education that need to be matched with possible implications on the country’s educational culture as well as ethical concerns and an investment in teachers’ training to ensure that the use of AI in education is equitable and accountable (Estrellado & Miranda, 2023). The rapid rise of GAI applications, such as ChatGPT, Microsoft Bing Chat, and Google Bard, has sparked considerable interest in their potential to transform teaching and learning in the Philippines. Researchers have begun to explore both the acceptability and impact of these technologies in various educational settings. Locally, studies have focused on diverse stakeholder groups ranging from senior high school teachers to college students and university administrators. Arguson et al. (2023) investigated the acceptability of GAI tools among senior high school teachers in Manila using the Technology Acceptance Model (TAM). Their study revealed that key factors such as self-efficacy and perceived ease of use significantly influenced teachers’ behavioral intention to adopt tools like Microsoft Bing Chat. The findings suggest that when teachers feel confident in their ability to use such tools, and when the tools are user-friendly, the likelihood of adoption increases. Similarly, a study by Guipitacio et al. (2025) revealed that AI programs significantly boost students self-confidence. Complementing these findings, Agbong-Coates (2024) examined the impact of ChatGPT integration on personalized learning outcomes among college students. Employing regression analysis and an omnibus ANOVA test, the study demonstrated that ChatGPT integration accounted for approximately 88.54% of the variability in personalized learning experiences. This indicates a strong relationship between AI use and improved individual learning outcomes. Notably, while demographic factors such as age, sex, and educational level had minimal effects, the significant influence of ChatGPT suggests that AI- driven personalized learning may be a powerful tool for enhancing academic performance in higher education. The insights from this study provide a valuable complement to classroom-focused research, highlighting the multifaceted impact of AI across the educational spectrum (Giray et al., 2024). The research on senior high school teachers revealed that ease of use and self-efficacy are crucial for AI tool adoption, while the college-level study confirms the substantial benefits of AI in personalized learning. Moreover, reflections from higher education administrators underscore the need for holistic approaches that address both instructional and operational challenges. These insights suggest that with proper training, infrastructure, and policy support, GAI has the potential to revolutionize educational practices in the Philippines, improving both teaching effectiveness and learning outcomes. MATERIALS AND METHODS This study is a quantitative research that uses focused- objective measurements through surveys/assessments. Utilizing a Wilcoxon Signed Rank-test for the data analysis. The population of the study are the Grade 11 STEM students. This research used a one-group pretest- posttest design, a type of quasi-experimental design that aims to evaluate causal relationships between intervention and outcome (Marsden & Torgerson, 2012). A private institution in Malabon City, Metro Manila, Philippines, was the chosen research locale of this study. The institution has proven academic excellence numerous times. Its competitive environment at the senior high school level makes it an ideal place for data gathering. With the innovative trends in the AI field, students are able to cope with the ever-changing environment. The researchers of this study utilized a purposive sampling approach, which is a non-probability type of sampling. Purposive sampling enables accurate and reliable data as it will be focused only on the STEM group of Grade 11. Leading to lowered randomization bias, because other academic strands exclude General Chemistry as their specialized course. Upon the inclusion of participants, only 30 had passed the criteria for the data analysis. Demographics of the participants consist of 15 boys and 15 girls. Both contribute to 50% of the total number of participants in the study. The General Chemistry course for participants is crucial for meeting the study’s objectives. Assessments in this subject involve identifying, solving problems, and illustrating compounds. Listed below is the identification of the participants that will undergo the study. Grade 11 STEM Students They specialize in the science course among other strands, encompassing insight into the concepts of General Chemistry. The respondents are addressed with a pretest without having prior knowledge in the stated topics. To collect the data for the study’s objective, the researchers employed the following: a researcher-made cognitive assessment and an adapted 5-point Likert scale TAM survey. Both the researcher-made assessment and worksheets cover the three following subtopics: Basic Subatomic Particles (BSP), Atomic Models (AM), and Pa ge 5 https://journals.e-palli.com/home/index.php/ajet Am. J. Educ. Technol. 4(3) 1-11, 2025 Atomic Composition (AM). These competencies are taken from MELCs from the Department of Education for Grade 11 STEM Students. The cognitive assessment will undergo an evaluation form that is adapted from Morales, 2003. An objective multiple choice is specialized in this form, limited only to the Atomic Structure topic. Both the cognitive test assessment and TAM survey had undergone pilot testing. Illustrated below are the results of Cronbach’s alpha value from the cognitive test and TAM survey. Accumulating a 0.7 and 0.8 Cronbach’s value translates to a reliable test. With the help of these instruments, the researchers will determine the effect of ChatGPT 3.0 on knowledge retention, critical thinking, and engagement Table 1: Accumulated Cronbach’s Alpha Value from the Pilot Testing Cognitive Assessment TAM Survey Cronbach’s Alpha Value 0.704 0.857 Figure 1: The Data Gathering Flow Chart through score rating. The completion of the pilot testing is followed by the distribution of the letter to the school administrators. Research, data gathering steps, and adherence to the required ethics is approved by the school principal, science coordinator, and general chemistry teacher. Distribution of the assent form for all the students is followed. Most of the students in the Grade 11 STEM are below 18 years of age. Thus, it is a crucial step in data gathering. All of the collected assent form from the students who rejected the study’s data analysis inclusion is eliminated. Yet, it is important to note that those students had still undergone the GAI intervention. A pretest is administered which acts as the baseline score of the participants. The worksheet will be answered with the assistance of ChatGPT 3.0. This consists of multiple choice (objective questions) that focus on the BSP, AM, and AC. Worksheet 1, focuses on the BSP and AM, while worksheet 2 for the AC. Post-test is administered to the students to measure their learning upon the usage of ChatGPT 3.0. The pretest and post-test consists of identical questions. However, the post-test consists of shuffled questions. Followed by the TAM, which focuses on the effect on the participants’ acceptance of technology. This consists of three subparts which are: Perceived Usefulness (PU), Perceived Ease of Use (PEU), and Behavioral Intention (BI). On the other hand, the criteria for the inclusion that should be complied with by the students are: shall (1) had taken the pretest and post-test, (2) had agreed to the assent form, (3) had taken the worksheet 1 and 2 with the use of ChatGPT 3.0, (4) had taken the TAM survey. After completing the following criteria they would be qualified for the data analysis phase of the study. The total number of qualified participants is n = 30. The large sample size will negate the disadvantageous implication of having a relatively low sampling consisting of a n<30 sample size. Illustrated below is the total number of participants in the Grade 11 STEM strand. Recorded responses are stored in an Excel file, ensuring readability and clarity to individual participant responses. Preserving the anonymity and confidentiality agreement to protect participants’ data is a requirement. The Table 2: Qualified Participants from the Grade 11 STEM Students n Qualified Participants 30 researchers encountered students who had limited internet access, had limited resources, or sick students during the data-gathering phase. To address these limiting variables, the researchers encouraged the sharing of internet access among the student(s). For the students who are sick, they are excluded from the study. Executing this systematic data collection ensures the most structured and reliable data from the participants. Data Analysis Procedure The study chose JAMOVI in order to calculate the reliability value of the cognitive assessment instrument. Utilizing the following data analysis tools such as the Wilcoxon Signed Rank-test, Cohen’s D Effect Size, and Hake’s Value Index to provide statistical data and derive conclusions on the stated research questions above. To answer RQ 1, the researchers will utilize Wilcoxon Signed Rank-test, a non-parametric statistic commonly employed in a quasi-experimental implemented in a non-randomized pretest and post-test. This is used if the central tendency (mean, median, and mode) of the Pa ge 6 https://journals.e-palli.com/home/index.php/ajet Am. J. Educ. Technol. 4(3) 1-11, 2025 accumulated data is set apart. Paired with a Cohen’s D Effect size value, which translates to having a small (value ≤ 0.2), medium (value ≤ 0.5), or large (value ≥ 0.8) effect on the participants (Smith et al. 2023). All of which are assisted by JAMOVI and Excel. Similarly, the RQ 2 utilized a Hake’s Gain Value Index. Deriving conclusions on the gain of the students on each subpart of the assessment such as: the BSP, AM, and AC. The stated index is classified from low gain (value ≤ 0.3), medium gain (value < 0.7), and high (value ≥ 0.7) gain. Lastly, the researchers investigated the mean and mode of the TAM survey data in order to answer the RQ 3 of this study. The values are supplemented by illustrations and texts in order to better understand the context. RESULTS AND DISCUSSION Pretest & Post-test Descriptive Statistics The mean, median, and mode of the two cognitive assessments resulted in a Shapiro-Wilk p-value of < 0.05. Two of the central tendencies (mean and median) have a close proximity for both pretest and post-test. However, the occurrence of a lower value in the mode resulted in the utilization of a non-parametric test. The pretest score accumulated a range value of 10, while the post-test was 11. A total of 30 participants are qualified in the data analysis of the study. Pretest & Post-test Results The statistical p-value is < 0.05 for the Wilcoxon Signed Rank test. Having a Cohen’s D effect size value of 0.9. Both of the results show a positive effect upon the Table 3: Central Tendency of the Cognitive Assessment Descriptives Pretest Post-test N 30 30 Mean 6.53 9.43 Median 6 9.5 Mode 5.00 a 6.00 a Standard deviation 2.21 3.35 Minimum 3 5 Maximum 13 16 Shapiro-Wilk W 0.917 0.927 Shapiro-Wilk p 0.023 0.04 Table 4: Non-parametric Results of the Cognitive Assessment Statistic df p Effect Size Pretest Post-test Student's t 5.22 29.0 <.001 Cohen’s d 0.935 Wilcoxon W 316a <.001 Rank biserial correlation 0.945 intervention of ChatGPT 3.0 in the General Chemistry course assessment. Does Generative AI Affect the Participants Assessment Score Related to Their Pretest Score after the Intervention? The results indicate a statistically significant improvement in students’ performance after using ChatGPT 3.0 as a learning aid. Wilcoxon Signed Rank Test yielded a p-value of < 0.05, confirming the increase in the post- test scores. Further being supported by the effect size value of (Cohen’s d = 0.9) that suggests a strong impact, indicating that the AI-assisted intervention contributed meaningfully to learning outcomes. Showing that they have learned throughout the intervention. The user-friendly nature of this algorithm through outputs led to the understanding of the concepts. The conceptual framework of this study depicts this effect upon the use of ChatGPT 3.0 as it encompasses scaffolding of complex concepts (Baidoo-Anu & Ansah, 2023) and improved cognitive skills (Yilmaz & Yilmaz, 2023). Cognitive learning was showcased in the BSP and AM part of the pretest and post-test. While the AC showed improvement in the problem-solving skills among the participants. This is supported by the study from recent works of literature that GAI has critical thinking, and problem-solving skills among students (Vazquez-Cano et al., 2021; Chang et al., 2022; Baidoo- Anu & Owusu Ansah; 2023). Conceptual understanding among the subparts prior to the intervention has resulted in a better assessment performance. ChatGPT 3.0 was able to teach the fundamentals of the BSP, AM, and AC among the participants. Which, ultimately led to an increase in their test performance. Assessment Subtopics Gain Value Three of the subtopics in the assessment had accumulated a Hake’s Gain value of less than 0.3. The BSP and AM have a close proximate value. While the AC’s accumulated value is higher. How Does Generative Al Affect the Participants Performance in the Basic Subatomic Particles (BSP), Atomic Model (AM), and Atomic Composition (AC) Subpart of the Assessment after the Intervention? Table 5: Hake’s Gain Value of the Cognitive Assessment Subtopics BSP AM AC Hake’s Gain Value 0.17 0.18 0.26 Pa ge 7 https://journals.e-palli.com/home/index.php/ajet Am. J. Educ. Technol. 4(3) 1-11, 2025 Hake’s Gain values for specific subtopics were low (< 0.3), suggesting modest conceptual retention in areas of BSP, AM, and AC. Students may have relied on ChatGPT3.0 for quick answers rather than engaging in higher-order cognitive processing, a pattern observed in previous AI-assisted learning studies (Yamaguchi et al., 2021). Furthermore, the short duration of the intervention, which may not have allowed sufficient time for deep conceptual learning. This is consistent with the study from Sun and Zhou (2024), which stated that the intervention duration of more than 11 weeks resulted in a high positive gain. Comprising only a 3-day intervention had less grasped the potential of ChatGPT 3.0’s ability in teaching the participants. Being one of the study’s recommendations for future research. Collected data from BSP and AM are significantly lower than AC. Showing that ChatGPT 3.0 improved the participants ability for problem-solving, while lesser for enhanced cognitive learning. Due to the 1-day intervention for all of the subparts, the participants had a hard time retaining the known knowledge while showing greater improvement on the problem-solving part of the post-test. TAM Survey Mean & Mode Value There are three primary subtopics included in the TAM survey which are PU, PEU, and BI. The given value of the mode is higher than the calculated mean. This corresponds to the fact that most of the participants had answered and strongly agreed in the TAM survey. How are the Participants Perceived Ease of Use (PEU), Perceived Usefulness (PU), and Behavioral Intention (BI) Influenced by Generative AI after the Intervention? Table 6: Tabulated Value of the TAM Survey PU PEU BI Mean Value 4.10 4.15 3.43 Mode Value 5 5 3 The PU and PEU upon the intervention of ChatGPT 3.0 resulted in the majority of the respondents answering 5, which corresponds to “strongly agree”. Both are above the mean score of 4.10 and 4.15, respectively. Suggesting that the algorithm’s usefulness and ease of use was greatly appreciated by the students. Further supported by the conceptual framework of this study, showing that assessment performance affects the PEU and PU of the participants. On the other hand, the Behavioral Intention towards the use of GAI was only 3, which corresponds to a neutral answer. BI’s mean value shows that its accumulated mode value is lower. There is a significant effect on the BI of the participants, but shows that it is in the middle value only. This is supported by the study from Gruenhagen et al. (2024) which noted that technological acceptance of GAI is expected upon its usage on academic tasks. PU, PEU, and BI is greatly affected by the intervention of ChatGPT 3.0 after the post-test. Most of the participants reported an above-mean response towards the use of ChatGPT 3.0. CONCLUSIONS This study investigated the effect of ChatGPT 3.0 in answering Atomic Structure assessments. Table 4 shows that utilization of ChatGPT 3.0 resulted in improvement of test scores among the participants. Its Cohen’s D value emphasized a large effect, which supplements the accumulated p-value. On the other hand, the Hake’s Gain value translates to a low gain in the following subtopics of the Atomic Structure assessment: BSP, AM, and AC. Participants had a hard time retaining the important concepts as learning of the following subtopics are only given a 3-day period. This led to understanding the concepts but having only a modest retention. PU and PEU from the TAM survey correspond to a mode of 5 on the likert scale. Showing that most of the participants had a positive influence after the utilization of ChatGPT 3.0. While having an accumulated mode value of 3, which corresponds to a neutral possibility for its utilization in future assessments. Button et al. (2013) stated that having >29 participants in a study reduces its possibility of attaining insignificant results. In this study, an equal of 30 participants are qualified, which lessened the possibility of yielding unreliable results. Future research should prioritize having a >11 week intervention period, explore other topics that have a standardized assessment, and utilize >30 participants to further explore the extent of ChatGPT 3.0 on General Chemistry topics. REFERENCES Agbong-Coates, I. J. G. (2024). ChatGPT Integration Significantly Boosts Personalized Learning Outcomes: A Philippine study. International Journal of Educational Management and Development Studies, 5(2), 165–186. https://doi.org/10.53378/353067 Alayacyac, J. R. S., Regidor, J. C., Caballo, J. H. S., Abellanosa, J. E., & Monaghan, G. J. G. (2024). Computer Self-Efficacy and Effectiveness of Quipper Learning Management System. American Journal of Smart Technology and Solutions, 3(1), 17–21. https://doi. org/10.54536/ajsts.v3i1.2428 Ali, Z., & Bhaskar, S. (2016). Basic Statistical Tools in Research and Data Analysis. Indian Journal of Anaesthesia, 60(9), 662. https://doi.org/10.4103/0019- Pa ge 8 https://journals.e-palli.com/home/index.php/ajet Am. J. Educ. Technol. 4(3) 1-11, 2025 5049.190623 Andrade, C. (2020). The Inconvenient Truth about Convenience and Purposive Samples. Indian Journal of Psychological Medicine, 43(1), 86–88. https://doi. org/10.1177/0253717620977000 Ardyansyah, A., Yuwono, A. B., Rahayu, S., Alsulami, N. M., & Sulistina, O. (2024). Students’ perspectives on the application of a generative pre-trained transformer (GPT) in chemistry learning: a case study in Indonesia. Journal of Chemical Education, 101(9), 3666-3675. https://doi.org/10.1021/acs. jchemed.4c00220 Arguson, A., Mabborang, M., & Paculanan, R. (2023). The acceptability of generative ai tools of selected senior high school teachers in schools divisions office-manila, philippines. Journal of Artificial Intelligence, Machine Learning and Neural Network, 3(6), 1-10. https://doi.org/10.55529/jaimlnn.36.1.10 Bai̇doo-Anu, D., & Ansah, L. O. (2023). Education in the Era of Generative Artificial Intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Journal of AI, 7(1), 52–62. https://doi.org/10.61969/jai.1337500 Bandi, A., Adapa, P. V. S. R., & Kuchi, Y. E. V. P. K. (2023). The power of generative ai: A review of requirements, models, input–output formats, evaluation metrics, and challenges. Future Internet, 15(8), 260. https://doi. org/10.3390/fi15080260 Baroni, I., Calegari, G. R., Scandolari, D., & Celino, I. (2022). AI-TAM: a model to investigate user acceptance and collaborative intention in human-in- the-loop AI applications. Human Computation, 9(1), 1-21. https://doi.org/10.15346/hc.v9i1.134 Baum, Z. J., Yu, X., Ayala, P. Y., Zhao, Y., Watkins, S. P., & Zhou, Q. (2021). Artificial intelligence in chemistry: current trends and future directions. Journal of Chemical Information and Modeling, 61(7), 3197-3212.https://doi. org/10.1021/acs.jcim.1c00619 Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2021). Introduction to meta-analysis. John wiley & sons. https://doi.org/10.1002/9780470743386 Bourban, M., & Rochel, J. (2020). Synergies in Innovation: Lessons Learnt from Innovation Ethics for Responsible Innovation. Philosophy & Technology, 34(2), 373–394. https://doi.org/10.1007/s13347- 020-00392-w Bozkurt, A., Junhong, X., Lambert, S., Pazurek, A., Crompton, H., Koseoglu, S., ... & Romero-Hall, E. (2023). Speculative futures on ChatGPT and generative artificial intelligence (AI): A collective reflection from the educational landscape. Asian Journal of Distance Education, 18(1), 53-130. https:// doi.org/10.5281/zenodo.7636568 Button, K. S., Ioannidis, J. P., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S., & Munafò, M. R. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature reviews neuroscience, 14(5), 365-376.5 Campbell, S., Greenwood, M., Prior, S., Shearer, T., Walkem, K., Young, S., ... & Walker, K. (2020). Purposive sampling: complex or simple? Research case examples. Journal of research in Nursing, 25(8), 652- 661. https://doi.org/10.1177/1744987120927206 Clark, J. M., & Paivio, A. (1987). A Dual Coding Perspective on Encoding Processes. In Springer eBooks. 5–33. https:// doi.org/10.1007/978-1-4612-4676-3_1 Clark, J. M., & Paivio, A. (1991). Dual Coding Theory and Education. Educational Psychology Review, 3(3), 149–210. https://doi.org/10.1007/bf01320076 Clark, M. (2021). What is Dual Coding Theory and How Can it Help Teaching?. CENTURY. https://www.century. tech/news/what-is-dual-coding-theory-and-how- can-it-help-teaching/ Clark, T. M. (2023). Investigating the Use of an Artificial Intelligence Chatbot with General Chemistry Exam Questions. Journal of Chemical Education, 100(5), 1905– 1916. https://doi.org/10.1021/acs.jchemed.3c00027 Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum. Danili, E., & Reid, N. (2006). Cognitive Factors that can Potentially Affect Pupils Test Performance. Chemistry Education Research and Practice, 7(2), 64–83. https://doi. org/10.1039/b5rp90016f Das, B., Majumder, M., Phadikar, S., & Sekh, A. A. (2021). Automatic question generation and answer assessment: a survey. Research and Practice in Technology Enhanced Learning, 16(1), 5. https://doi.org/10.1186/ s41039-021-00151-1 David, H. A., & Gunnink, J. L. (1997). The Paired T-test Under Artificial Pairing. The American Statistician, 51(1), 9–12. https://doi.org/10.1080/00031305.199 7.10473578 Department of Education, Philippines. (n.d.). Academic track STEM. Retrieved January 10, 2025. DepEd Computerization Program | Department of Education. (n.d.). Retrieved January 10, 2025, from https://www.deped.gov.ph/2018/04/06/deped- computerization-program/ Doe, J. K. (n.d.). Toward a Firm Technology Adoption Model (F-TAM) in A Developing Country Context. AIS Electronic Library (AISeL). Retrieved October 8, 2024, from https://aisel.aisnet.org/mcis2017/23/ Du Plessis, W. P. (2020). Automated Generation of Test Questions and Solutions. 2020 IFEES WorldEngineering Education Forum - Global Engineering Deans Council (WEEF-GEDC) (pp. 1–5). https://doi.org/10.1109/ weef-gedc49885.2020.9293635 E. Morales, M. P. (2006). Development and Validation of a Concept Test in Introductory Physics for Biology Students. De La Salle University. https://www.dlsu.edu.ph/ wp-content/uploads/pdf/research/journals/mjs/ MJS07-2-2012/MJS07-2-4-morales.pdf Editor & Editor. (2024). The Role of Generative AI in Education: Use Cases, Benefits and Challenges in 2024. Fullestop Blogs. https://www.fullestop.com/blog/generative-ai-in- education-use-cases-benefits-and-challengs Pa ge 9 https://journals.e-palli.com/home/index.php/ajet Am. J. Educ. Technol. 4(3) 1-11, 2025 Eke, D. O. (2023). ChatGPT and the Rise of Generative AI: Threat to Academic Integrity? Journal of Responsible Technology, 13, 100060. https://doi.org/10.1016/j. jrt.2023.100060 Estrellado, C. J., & Miranda, J. C. (2023). Artificial Intelligence in the Philippine Educational Context: Circumspection and Future Inquiries. Social Science Research Network. https://papers.ssrn.com/sol3/ papers.cfm?abstract_id=4442136 Farrelly, T., & Baker, N. (2023). Generative Artificial Intelligence: Implications and Considerations for Higher Education Practice. Education Sciences, 13(11), 1109. https://doi.org/10.3390/educsci13111109 García-Martínez, I., Fernández-Batanero, J. M., Fernández-Cerero, J., & León, S. P. (2023). Analysing the impact of artificial intelligence and computational sciences on student performance: Systematic review and meta-analysis. Journal of New Approaches in Educational Research, 12(1), 171-197. https://doi. org/10.7821/naer.2023.1.1240 Gasteiger, J. (2020). Chemistry in Times of Artificial Intelligence. ChemPhysChem, 21(20), 2233–2242. https://doi.org/10.1002/cphc.202000518 Giray, L., De Silos, P. Y., Adornado, A., Buelo, R. J. V., Galas, E., Reyes-Chua, E., ... & Ulanday, M. L. (2024). Use and Impact of Artificial Intelligence in Philippine Higher Education: Reflections from Instructors and Administrators. Internet Reference Services Quarterly, 28(3), 315–338. https://doi.org/10.1080/10875301. 2024.2352746 Gray, N. (1988). Artificial Intelligence in Chemistry. Analytica Chimica Acta, 210, 9–32. https://doi. org/10.1016/s0003-2670(00)83874-x Gruenhagen, J. H., Sinclair, P. M., Carroll, J. A., Baker, P. R., Wilson, A., & Demant, D. (2024). The Rapid Rise of Generative AI and its Implications for Academic Integrity: Students’ Perceptions and Use of Chatbots for Assistance with Assessments. Computers and Education Artificial Intelligence, 7, 100273. https://doi. org/10.1016/j.caeai.2024.100273 Guipitacio, J. A. D., Aleman, A. V. B., Bonsubre, C. M., Galleto, J. M. T., Tapere, B. N. B., Caballo, J. H., & Caangay, R. B. (2025). The Nexus Between AI Self- Efficacy and Attitude Towards AI of University Students in Davao City as Moderated by Sex. American Journal of Smart Technology and Solutions, 4(1), 1–7. https://doi.org/10.54536/ajsts.v4i1.3788 Hadžibegović, Z., & Sulejmanović, S. (2014). Fundamental Thermal Concepts Understanding: The First-year Chemistry Student Questionnaire Results. Bulletin of the Chemists and Technologists of Bosnia and Herzegovina, 42, 21-30. Hashmi, N., & Bal, A. S. (2024). Generative AI in Higher Education and Beyond. Business Horizons. https://doi. org/10.1016/j.bushor.2024.05.005 Hattie, J. (2009). Visible learning: A synthesis of over 800 meta- analyses relating to achievement. Routledge. He, L., Bai, L., Dionysiou, D. D., Wei, Z., Spinney, R., Chu, C., ... & Xiao, R. (2021). Applications of Computational Chemistry, Artificial Intelligence, and Machine Learning in Aquatic Chemistry Research. Chemical Engineering Journal, 426, 131810. https://doi. org/10.1016/j.cej.2021.131810 Hedberg, E., & Ayers, S. (2014). The Power of a Paired T-test with a Covariate. Social Science Research, 50, 277–291. https://doi.org/10.1016/j. ssresearch.2014.12.004 Honors Chemistry Test: Atomic History and Structure. (2015). Scribbd. https://www.scribd.com/doc/293180968/ atomic-structure-exam How Generative AI is Reshaping Education in Asia- Pacific. (2024). UNESCO. https://www.unesco.org/ en/articles/how-generative-ai-reshaping-education- asia-pacific Hutt, S., & Hieb, G. (2024). Scaling Up Mastery Learning with Generative AI - Exploring How Generative AI Can Assist in the Generation and Evaluation of Mastery Quiz Questions. University of Denver. https://doi. org/10.1145/3657604.3664699 ICT. (n.d.). Retrieved January 10, 2025, from www. depedimuscity.com. https://www.depedimuscity. com/services/ict.php Kabudi, T., Pappas, I., & Olsen, D. H. (2021). AI-enabled adaptive learning systems: A systematic mapping of the literature. Computers and education: Artificial intelligence, 2, 100017. https://doi.org/10.1016/j. caeai.2021.100017 Khan, I., Al Sadiri, A., Ahmad, A. R., & Jabeur, N. (2019, January). Tracking student performance in introductory programming by means of machine learning. In 2019 4th mec international conference on big data and smart city (ICBDSC) (pp. 1-6). IEEE.https:// doi.org/10.1109/ICBDSC.2019.8645608 Kim, T. K. (2015). T-test as a Parametric Test. Kamje. https://doi.org/10.4097/kjae.2015.68.6.540 Leelavathi, R., & Surendhranatha, R. C. (2024). ChatGPT in the Classroom: Navigating the Generative AI Wave in Management Education. Journal of Research in Innovative Teaching & Learning. https://doi. org/10.1108/jrit-01-2024-0017 Lim, W. M., Gunasekara, A., Pallant, J. L., Pallant, J. I., & Pechenkina, E. (2023). Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. The international journal of management education, 21(2), 100790. https://doi.org/10.1016/j.ijme.2023.100790 Lo, C. K. (2023). What is the Impact of CHATGPT on Education? A Rapid Review of Literature. Education Sciences, 13(4), 410. https://doi.org/10.3390/educsci13040410 Lowe, R. (2021). OpenAI’s GPT-2: The Model, the Hype, and the Controversy. Medium. https://towardsdatascience. com/openais-gpt-2-the-model-the-hype-and-the- controversy-1109f4bfd5e8?gi=2d3fca86e7c0 Lutkevich, B., & Schmelzer, R. (2023). GPT-3. Enterprise AI. https://www.techtarget.com/searchenterpriseai/ definition/GPT-3 Pa ge 10 https://journals.e-palli.com/home/index.php/ajet Am. J. Educ. Technol. 4(3) 1-11, 2025 Marangunić, N., & Granić, A. (2014). Technology Acceptance Model: A Literature Review from 1986 to 2013. Universal Access in the Information Society, 14(1), 81–95. https://doi.org/10.1007/s10209-014-0348-1 Marikyan, D. & Papagiannidis, S. (2023). Technology Acceptance Model: A Review. In S. Papagiannidis (Ed), TheoryHub Book. https://open.ncl.ac.uk / ISBN: 9781739604400 Marsden, E., & Torgerson, C. J. (2012). Single Group, Pre- and Post-test Research Designs: Some Methodological Concerns. Oxford Review of Education, 38(5), 583–616. https://doi.org/10.1080/03054985.2012.731208 Marshall, G., & Jonker, L. (2010). An Introduction to Descriptive Statistics: A Review and Practical Guide. Radiography, 16(4), 1–7. https://doi.org/10.1016/j. radi.2010.01.001 Metcalfe, L. (1996). The Investigation of Student Understanding of Atomic Structure and Bonding. Fui-Hoon Nah, F., Zheng, R., Cai, J., Siau, K., & Chen, L. (2023). Generative AI and ChatGPT: Applications, Challenges, and AI-human Collaboration. Journal of Information Technology Case and Application Research, 25(3), 277–304. https://doi.org/10.1080/15228053.2 023.2233814 Naseer, F., Khalid, M. U., Ayub, N., Rasool, A., Abbas, T., & Afzal, M. W. (2024). Automated assessment and feedback in higher education using generative AI. In Transforming Education With Generative AI: Prompt Engineering and Synthetic Content Creation (pp. 433- 461). IGI Global Scientific Publishing. https://doi. org/10.4018/979-8-3693-1351-0.ch021 Nick, T. G. (2007). Descriptive Statistics In: Ambrosius, W.T. (eds) Topics in Biostatistics Methods in Molecular Biology™, v. 404. Humana Press. https://doi. org/10.1007/978-1-59745-530-5_3 Paired T-tests. (n.d.). Coventry. Retrieved November 17, 2024 Paivio, A. (2018). Dual Coding Theory. InstructionalDesign. org. https://www.instructionaldesign.org/theories/ dual-coding/ Passi, S., & Vorvoreanu, M. (2024). Over Reliance on AI: Literature review - Microsoft Research. Microsoft Research. https://www.microsoft.com/en-us/research/ publication/overreliance-on-ai-literature-review/ Pesovski, I., Santos, R., Henriques, R., & Trajkovik, V. (2024). Generative AI for customizable learning experiences. Sustainability, 16(7), 3034. https://doi. org/10.3390/su16073034 PHL CHED Connect - We Educate as One. (n.d.). Phlconnect.ched.gov.ph. Retrieved January 10, 2025, from https://phlconnect.ched.gov.ph/content/view/ governance-of-artificial-intelligence Raftery, D. (2023). Will ChatGPT Pass the Online Quizzes? Adapting an Assessment Strategy in the Age of Generative AI. Irish Journal of Technology Enhanced Learning, 7(1). https://doi.org/10.22554/ ijtel.v7i1.114 Rahman, M. A., Kabir, M. A., Haque, M. E., & Hossain, B. M. (2021). A Machine Learning-Based Price Prediction for Cows: http://doi. org/10.5281/ zenodo. 4817941. American Journal of Agricultural Science, Engineering and Technology, 5(1), 64-69. https:// doi.org/10.54536/ajaset.v5i1.63 Rai, N. (2016). A Study on Purposive Sampling Method in Research. Ksl. https://www.academia. edu/28087388/a_study_on_purposive_sampling_ method_ in_research Ramdurai, B. (2023). The Impact, Advancements, and Applications of Generative AI. ResearchGate. https:// www.researchgate.net/publication/371314493_ The_Impact_Advancements_and_Applications_of_ Generative_AI Schnotz, W., & Horz, H. (2010). New Media, Learning From. In Elsevier eBooks, p. 140–149. https://doi. org/10.1016/b978-0-08-044894-7.00740-5 Sigmoid. (2024). GPT-3: All You Need to Know About AI Language Models. Sigmoid. https://www.sigmoid. com/blogs/gpt-3-all-you-need-to-know-about-the- ai-language-model/ Su, J., & Yang, W. (2023). Unlocking the Power of ChatGPT: A Framework for Applying Generative AI in Education. ECNU Review of Education, 6(3), 355– 366. https://doi.org/10.1177/20965311231168423 Sun, L., & Zhou, L. (2024). Does Generative Artificial Intelligence Improve the Academic Achievement of College Students? A Meta-Analysis. Journal of Educational Computing Research, 62(7), 1896–1933. https://doi.org/10.1177/07356331241277937 Talikan, A. I., Salapuddin, R., Aksan, J. A., Rahimulla, R. J., Ismael, A., Jimlah, R., ... & Ajan, R. A. (2024). On paired samples T-test: Applications, examples and limitations. Ignatian, 2(4), 943-951. https://doi. org/10.5281/zenodo.10987546 Tamer, H., & Knidiri, Z. (2023). University 4.0: Digital Transformation of Higher Education Evolution and Stakes in Morocco. American Journal of Smart Technology and Solutions, 2(1), 20–28. https://doi.org/10.54536/ ajsts.v2i1.1300 Thanh, B. N., Vo, D. T. H., Nhat, M. N., Pham, T. T. T., Trung, H. T., & Xuan, S. H. (2023). Race with the machines: Assessing the capability of generative AI in solving authentic assessments. Australasian Journal of Educational Technology, 39(5), 59-81. https://doi. org/10.14742/ajet.8902 Tobler, S. (2023). Smart Grading: A Generative AI-based Tool for Knowledge-grounded Answer Evaluation in Educational Assessments. MethodsX, 12, 102531. https://doi.org/10.1016/j.mex.2023.102531 Tongco, M. D. C. (2007). Purposive Sampling as a Tool for Informant Selection. Scholar Space. http://hdl.handle. net/10125/227 Tsai, D. C., Huang, A. Y., Lu, O. H., & Yang, S. J. (2021, July). Automatic question generation for repeated testing to improve student learning outcome. In 2021 International Conference on Advanced Learning Technologies (ICALT) (pp. 339-341). IEEE. https:// doi.org/10.1109/icalt52272.2021.00108 Pa ge 11 https://journals.e-palli.com/home/index.php/ajet Am. J. Educ. Technol. 4(3) 1-11, 2025 Wang, Y., Tang, Y., & Ye, Z. S. (2022). Paired or partially paired two-sample tests with unordered samples. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(4), 1503-1525. https://doi.org/10.1111/rssb.12541 Wong, L. H., Chen, W., & Jan, M. (2012). How artefacts mediate small-group co-creation activities in a mobile- assisted seamless language learning environment?. Journal of Computer Assisted Learning, 28(5), 411-424. https://doi.org/10.1111/j.1365-2729.2011.00445.x Wood, D., & Moss, S. H. (2024). Evaluating the Impact of Students’ Generative AI Use in Educational Contexts. Journal of Research in Innovative Teaching & Learning. https://doi.org/10.1108/jrit-06-2024-0151 Manfei, X. U., Fralick, D., Zheng, J. Z., Wang, B., Xin, M. T., & Changyong, F. E. N. G. (2017). The differences and similarities between two-sample t-test and paired t-test. Shanghai archives of psychiatry, 29(3), 184. https:// doi.org/10.11919/j.issn.1002-0829.217070 Yilmaz, R., & Yilmaz, F. G. K. (2023). The Effect of Generative Artificial Intelligence AI-based Tool Use on Students’ Computational Thinking Skills, Programming Self-efficacy and Motivation. Computers and Education Artificial Intelligence, 4, 100147. https:// doi.org/10.1016/j.caeai.2023.100147 Zhang, L., & Xu, J. (2024). The Paradox of Self-efficacy and Technological Dependence: Unraveling Generative AI’s Impact on University Students’ Task Completion. The Internet and Higher Education, 65, 100978. https:// doi.org/10.1016/j.iheduc.2024.100978