Beem, H. Global Journal of Transformative Education (2020) Vol 2 DOI 10.14434/gjte.v2i1.31224 Open Access Published by the Global Insitutute of Transformative Education (http://www.gite.education) © Beem. 2020. Open Access This journal is distributed under the terms of the Creative Commons Attribution NonCommercial NonDerivative 4.0 International License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits unrestricted use, distribution, and reproduction without revision in any non-commercial medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, Effect of Hands-on Science Activities on Ghanaian Student Learning, Attitudes, and Career Interest: A Preliminary Control Study Heather Beem Abstract A quasi-experimental study was carried out with 309 Form 3 students across 9 public Junior High Schools in the Greater Accra Region of Ghana. The effect of Practical Education Network (PEN)’s approach of training STEM teachers to employ hands-on activities using low-cost, locally-available resources was studied in terms of student learning outcomes, attitudes towards learning science, and interest in STEM majors/ careers. Over a 2.5-month period, the science teacher at each experimental school received a weekly training on a hands-on activity and lesson observation by the respective PEN Trainer. A survey on attitudes towards science and a previous edition of the national exam (BECE) were administered to all students before and after the intervention. The mean pre-post differences were compared between the experimental and control schools. The intervention caused an average of 10.9% increase in exam scores (difference-in-differences), but the results were mixed at the school-level. Unpaired t-tests and Hedge’s g tests were used to determine statistical significance between the two groups. Student engagement increased significantly (p = 3 x 10-7, g = 0.85), and student enjoyment of science increased 22% more, on average. The intervention disproportionately affected the females positively, enabling greater learning gains (14.5% vs. 5.3% for the males), greater increase in engagement, and a significant shift in interest towards STEM majors and careers, which their male counterparts did not experience. Results from this study should inform the design of future studies with longer duration and which account for factors such as school infrastructure quality. Keywords: STEM, Ghana, Hands-on activities, Attitudes Full listing of authors and contacts can be found at the end of this article. Introduction In Ghana, students use the phrase “chew and pour, pass and forget” to describe their experience of learn- ing in school. This phrase expressively captures how students are asked to “chew” information, repeating facts over and over again, “pour” (vomit) them out on the exams, attempt to pass the exams, and then prompt- ly move on with their lives (Blench & Dendo, 2006, Quansah & Asamoah, 2019). The dominance of this phrase in the Ghanaian vernacular points to the wide recognition that a shift in pedagogical practice is need- ed. Ghanaian educationists have pointed out the detri- mental effects that “chew and pour” has on students’ creativity (Haffar, 2016) and ability to translate theory to useful outcomes (Adomako-Ampofo & Kaufmann, 2018). The future world of work in Africa is technology- based (World Economic Forum, 2017). For the growing youth population in Africa to rise to these demands, the education system needs to be able to engage students, drive deep learning, and build their interest in STEM. Ghanaian education stakeholders clearly express that a shift in teaching practice is key to achieving this. The new national curriculum states “Ghana believes that an effective science education needed for sustainable development should be inquiry-based” (Ministry of Education Ghana, 2019). Interventions that can create enduring, transformative change in STEM teaching in Ghana should be developed and tested. Practical Education Network (PEN) is an NGO seeking to shift the dominant pedagogical mode in West African STEM classrooms from rote to experiential. Sur- vey data collected from a few hundred Ghanaian public Junior High School (JHS) teachers reveals that virtually all teachers see the benefit of using hands-on activities, but 80% cite the lack of resources as the main challenge Beem, Effect of Hands-on Science Activities 19 they face in teaching more experientially. Furthermore, less than 5% reported having attended any relevant training towards this challenge within the last year (Practical Education Network, 2016). With minimal resources and training available, most Ghanaian teach- ers feel there are no realistic alternatives to the “chew and pour” approach. PEN is tackling this challenge by training science and mathematics teachers to employ low-cost, locally-available materials for the develop- ment and deployment of hands-on activities, which are aligned to the national curricula. The aim of this study is to determine the impact of PEN’s approach on stu- dents in the Ghanaian classroom. We hypothesize that regular use of these low-cost, hands-on techniques in the science classroom will improve Ghanaian students’ exam scores, attitudes towards learning science, and their interest in pursuing STEM in the future. Literature Review Research in science education has established strong positive effects when students are taught using experiential pedagogies. These approaches have been shown to enhance student attitudes (Gormally, Brick- man, Hallar, & Armstrong, 2009), improve exam scores (Abdi, 2014), increase scientific process skills (Ergul et al., 2011), and potentially encourage more students to pursue STEM-related careers (van den Hurk, Meelissen, & van Langen, 2019). The body of literature has largely been developed in the Global North, but a recent study (Bando et al., 2019) compiled the results of randomized controlled trials deployed across four Latin American countries, assessing the efficacy of the inquiry-based approach across a total of 17K students. Their results showed a 0.16 standard deviation increase in science test scores after 7 months of practical science learning. There is a pressing need to understand how to contextualize international best practices for African education, given the low learning outcomes presently being recorded here. In the early 2000’s, Ghana began participating in the Trends in International Mathemat- ics and Science Study (TIMMS). Ghana has continu- ally ranked near or at the bottom of the participating countries (Buabeng, Owusu, & Ntow, 2014). Despite Ghanaian education stakeholders’ recognition that improvements in learning outcomes are needed, only a few studies have been conducted to determine the efficacy of experiential pedagogies in the local science education context. One study at the senior high school level (Aboagye, 2009) compared the effectiveness of a particular constructivist approach (the three-phase learning cycle) with the traditional approach used in Ghanaian science classrooms. It was used in the context of teaching one specific topic (direct current electricity). In South Africa, Kibirige, Rebecca & Mavhunga (2014) studied 60 high school students, half of which were undergoing three weeks of experimental work (using standard laboratory equipment) and the other half which were undergoing traditional lecture methods. In both cases, they measured improvement on exam scores as a result of the practical sessions. These studies indicate that experiential pedagogies can improve learning outcomes in the African science classroom. More such studies should be done to understand details of implementation, and they should also be carried out at earlier levels of schooling. In Ghana, students in senior high school have already chosen a major of study. In order to understand and impact students’ career prospects, interventions and studies are needed at the primary and junior high school levels. Even if the efficacy of such pedagogies is established in the African context, the question of how to implement such approaches still remains. In Ghana, less than 10% of public junior high schools contain any laboratory equipment (S. Mohammed, personal communication, March 2015). For hands-on, experiential lessons to be widely deployed, teaching and learning materials must be low-cost. (Davis & Chaiklin, 2015) studied the use of classroom objects, such as tables and chairs, as teaching and learning resources for Ghanaian students to learn measurement. With over 500 hands-on activities made from materials available locally in Ghana (Practical Education Network, 2020), PEN’s content is one of the most extensive and relevant resources currently available to the Ghanaian science teacher. Its alignment with the Ghanaian national curriculum also warranted its infusion into the latest revision of the primary school science curriculum (Ministry of Education Ghana, 2019) and the accompanying Teacher Resource Pack’s list of “Practical Science Lesson Resources” (National Council for Curriculum & Assessment, 2019). In addition to the content itself, teacher training is a key component in enabling a shift from rote to experiential pedagogies. In Ghana, where teacher- centered approaches tend to dominate science teaching (Buabeng, Ossei-Anto, & Ampiah, 2014), teacher training has been pointed out as a key factor to improving student outcomes (Buabeng, Owusu, & Ntow, 2014). The details of how a teacher implements Global Journal of Transformative Education (2020) Vol 2 Beem, Effect of Hands-on Science Activities 20 Global Journal of Transformative Education (2020) Vol 2 practical content also affects the efficacy of the approach (Abrahams & Millar, 2008). Various teacher training interventions have been successfully carried out in Ghana, but they have mostly been focused on literacy and numeracy (Aizenman & Warner, 2018; Johnston & Ksoll, 2017). The role of gender as it relates to science education in Ghana has been subject to some investigation. (Donkor & Justice, 2016) sought to uncover the reasons behind the gender gap in students pursuing science in the Upper West Region of Ghana. Further research is needed to elicit key mechanisms that can close the gap. The study in South Africa mentioned above (Kibirige et al., 2014) found no difference in results across gender lines. This study seeks to understand the efficacy of Practical Education Network’s approach, which aims to tackle the aforementioned challenges through its STEM teacher training program that equips Ghanaian teachers to leverage local materials and carry out hands-on, experiential pedagogies in their classrooms. The research questions are as follows: 1. What effect does PEN’s approach have on learning outcomes? 2. What effect does PEN’s approach have on student attitudes to learning science? 3. What effect does PEN’s approach have on student interest in STEM majors and careers? Finally, any difference in results across gender and geographical (rural vs. urban) lines are to be elicited. Methodology Overview A quasi-experimental, quasi-controlled method was employed to measure the effect of training Ghanaian science teachers to use practical, hands-on activities in their Junior High School classrooms. Three PEN trainers were selected to lead the intervention. All three are public JHS science teachers who had attended PEN’s Introduction to Hands-on Science training program in the past and performed well enough to be invited for a second round of training, the completion of which promoted them to PEN Trainer status. Each PEN Trainer was enlisted to train 1-2 science teachers in their respective Circuits- the next geographic division below a District and usually composed of 5-20 schools. Those teachers were, in turn, prepared to deliver at least one hands-on activity per week in their classroom over a 2.5-month period. The study took place during the 3rd (final) term of the 2016/7 Academic Year at 9 public schools in various locations within the Greater Accra Region. In total, N = 309 students were involved in the study: 135 from the experimental schools and 174 from the control schools. The trainers solicited and received permission from each participating school to include information on their teachers and students in this study, All school, teacher and student names have been kept anonymous, but select pictures are shown to aid in depicting the intervention. School Selection The trainers were asked to select their own participating schools- both experimental schools and control schools. They were allowed to choose any schools so long as they were located within their Circuit. In all cases, the selection ended up being driven by 1) those which were most easily accessible for the trainers, in terms of distance to travel, and 2) those at which they were positively received by the headteachers. The author’s assumption was that all schools within the same Circuit would be similar socioeconomically and in terms of exam performance. The trainers were briefed on the goals of the study, including the intention that both experimental and control schools be similar. They were asked to use their knowledge of the schools to select those they deemed to be comparable. All of the schools are located in the Greater Accra Region of Ghana. Two of the Circuits are located in rural areas (Kofi Kwei and Ashalaja - both within the Ga South District) and one Circuit is in an urban area (Kwabenya - within the Ga East District). The total number of experimental schools is five and that of control schools is four. Table 1 provides the code Table 1. List of participating schools Beem, Effect of Hands-on Science Activities 21 Global Journal of Transformative Education (2020) Vol 2 used to refer to each school in the course of this study, their type (experimental: “Exp” or control: “Ctrl”), the location of the school (Urban or Rural), and the total number of students engaged in each school. In each case, all of the students in the final year (Form 3) class were engaged in the study. This Form was chosen, as their students would be taking the terminal exam for JHS, Basic Education Certificate Examination (BECE), shortly after the end of the study. Tools Administered After permission was solicited from the Ghana Education Service (GES) District Offices and the schools were selected, a pre-test was administered to the students at all 9 schools (experimental and control). This pre-test was used to determine a baseline for the three main outcomes of interest in this study: student exam scores in science, student attitudes towards learning science, and student interest in STEM majors and careers. One tool was used to measure the first outcome and a second tool was used to measure the other two outcomes. The first tool, which was used to assess the first research question, was the full Integrated Science portion of a previous Basic Education Certification Examination (BECE). Nearly all students write this exam in June, but a second version of the exam is offered in the following February for a small minority of students who require writing it then. The February 2017 version of the BECE is the tool which was administered to the students participating in this study. This was at the suggestion of a former examiner for the West African Examinations Council (WAEC), the body overseeing the national exams, as he believed that few to none of the students in this study would have seen that particular exam. That person also created the marking scheme used for assessment in this study. The second tool administered was a paper survey, which included the following survey questions (SQ): SQ1. Which of the following subjects do you intend to study in Senior High School (SHS)? SQ2. What job are you most interested in working after you leave school? SQ3. How frequently does your science class include hands-on activities? SQ4. How engaged are you in the hands-on activities? SQ5. How easy is it to learn science? SQ6. Do you enjoy learning science? SQ7. If you do, why? If not, why not? The second research question (students’ attitudes towards learning science) was measured via SQ4-7. The third research question (students’ interest in pursuing STEM majors and careers) was measured via SQ1-2. Finally, SQ3 was simply used as a check for whether the experimental schools were indeed receiving the intervention. For SQ1, all potential subjects that students have as options for their SHS “major” were listed. SQ2 and SQ7 were open-ended responses. Options for SQ3 included ”Never”, “Once per month”, “Once per week”, and “More than once per week”. SQ4 and SQ5 were answered on a Likert scale from 1 to 7. SQ6 was a yes/no question. Both the survey and the exam were administered in paper form to the students. PEN staff administered the surveys so as to best enable students to provide honest feedback in the absence of their teacher. The trainers administered the exams. At the end of this period, similar tools were administered to the students to serve as the endline. The survey was the exact same as that administered at the beginning of the intervention. The exam administered at the end was, however, a different version. The post-test exam was shorter, composing only Part 1 of the two parts composing the BECE. It was created by compiling questions from different years’ versions of Part 1 of the Integrated Science portion of the BECE. Given the short duration of the intervention, only a portion of the syllabus was covered. The selection of past questions ensured that the syllabus topics covered by both sets of teachers would appear on this exam. The selection was done by the research team and checked for fairness by the WAEC examiner mentioned above. Both experimental and control schools completed this endline exam. One intern completed the marking of all exam scripts so as to ensure uniformity. Intervention (Training + Lesson Observation) Once the pre-test (survey and exam) had been administered at all 9 participating schools, a 2.5-month period of the intervention commenced at the 5 experimental schools. The intervention consisted of the following steps, was repeated on a weekly basis, and is also depicted schematically in Figure 1. 1. The experimental school teacher notified PEN’s Logistics Officer of the science curriculum topic they would be treating in the next week. 2. PEN’s Logistics Officer procured locally-available materials for a PEN hands-on science activity Beem, Effect of Hands-on Science Activities 22 corresponding to that topic, and he delivered the materials to the PEN trainer. 3. The PEN trainer trained the experimental school teacher on PEN’s hands-on science activity for that syllabus topic. 4. The teacher delivered the practical lesson to their Form 3 students, while the PEN trainer observed. 5. The PEN trainer provided feedback to the teacher on their lesson delivery. The PEN trainer visited the experimental school(s) in their Circuit twice a week- once to train the teacher and once to observe the deployment of the practical lesson. This cycle continued every week in the experimental schools, while the control schools were exempted. Figure 2 shows a picture of one of the teacher training sessions and trainer observation of the lesson on heat energy. Figure 3 shows the PEN activity used to teach the digestive system, as carried out by students at one of the experimental schools after their teacher had been trained on the lesson. Data Analysis Research Question 1 – student exam scores: In order to determine the level of effect that the intervention had on the exam scores, a difference-in- differences analysis was carried out between the means of the experimental and control groups. Pre-test and post-test exam scores are first presented independently. Global Journal of Transformative Education (2020) Vol 2 Figure 1. The process flow completed weekly over the course of the intervention. Figure 2. Teacher training and lesson monitoring of a PEN activity on heat energy transfer through painting of Milo tins. Beem, Effect of Hands-on Science Activities 23 whose responses were captured in both the pre and post surveys were considered in this analysis. Before carrying out the t-test, an F-test was carried out to de- termine whether each dataset was of equal or unequal variance. The final method was included as an addition- al measure of the magnitude of difference between the two groups due to its utility in working with smaller sample sizes. The effect size is considered small if g ≥ 0.2, medium if g ≥ 0.5, and large if g ≥ 0.8. For Research Question 3, the potential responses to the corresponding survey questions (SQ1 - SHS major and SQ2 - future job interest) were first categorized into STEM or non/STEM responses. For SQ1 (SHS major), STEM responses were considered to be “Gen- eral Science”, “Agric Science” or “Other” if the response included something in the line of “Technical” or “Elec- tricals”. For SQ2 (future job interest), STEM responses were ones such as “engineer”, “doctor”, “nurse”, “tech- nician”, “accountant”, “pharmacist”, “architect”, etc. The survey results were digitized and analyzed in Excel. For Q6 which has a binary choice of answers, the number of respondents in each category was counted before and after the intervention. The pre-post difference in number of respondents in each category was compared between the experimental and control schools. For the questions which were answered on a scale (Q3-5), statistical analysis was done to determine the signif- icance of any difference in the means over time. Both the means between the pairs of schools and the means between the experimental and control schools were compared. The schools in the Ashalaja Circuit (ES2, ES3, CS2) were excluded from this analysis due to the logisti- cal challenges of the trainer falling ill and not being able to administer the post-survey at all of his schools. The same statistical analyses were carried out with the data disaggregated across gender and geography, with the goal of elucidating any effects that may have been felt more strongly within either the female or male subsets and within either the rural or urban subsets. Confirmation of activity implementation In addition to information received from the PEN Trainers regarding details of the intervention im- plementation, the results from Survey Question 3 (“How frequently does your science class include hands-on activ- ities?”) were used to determine whether the hands-on science activities had indeed been delivered regularly at the experimental schools. Using the same statistical analyses described for the above (t-test, Hedge’s g), the difference in results over the course of the interven- tion was compared across the combined experimental schools and combined control schools. Follow-up school visits After this analysis was done, follow-up visits to the schools were carried out. The results to the study’s Research Questions were presented to the teachers and trainers involved, and their interpretation of the find- ings were elicited. Quotes were collected and key ones are reported here. Logistical Challenges The trainer for the Ashalaja Circuit fell severely ill partway through the study. He was therefore unable to carry out the full intervention at his set of experimen- tal schools. He was also unable to be reached after the study, so the exact number of trainings he ran remains unknown. For this reason, the only results included for the Ashalaja Circuit schools (ES2, ES3, CS2) are the pre- test (exam and survey). Those schools were excluded from all analyses involving a comparison over time. Global Journal of Transformative Education (2020) Vol 2 Figure 3. The PEN model of the digestive system, utilizing locally-available resources, constructed by students at ES2. The difference in the means over time is presented last, and only includes students who completed both pre- test and post-test exams. Research Questions 2&3 – student attitudes to learning science; student interest in STEM majors/ careers: Three main methods were used to answer these research questions through comparison of the experi- mental and control groups: 1) comparison of yes/no responses over time, 2) two-tailed, unpaired t-test to determine the p-value using a 5% significance level, 3) test for effect size using the Hedges’ g statis- tic. Only those students Beem, Effect of Hands-on Science Activities Activities 24 Results Confirmation of activity implementation Results from SQ3 “How frequently does your science class include hands-on activities?” (scale of 1 to 4) are shown in Table 2, with sample size (N) included. Overall, the experimental schools reported significantly higher increase (Δ) in frequency of including hands- on activities compared to that of the control schools over the course of the intervention (p = 0.007). A small effect size (g = 0.47) was also measured. This serves as a confirmation that, on average, the trainings were successfully being translated to classroom implementation in the two Circuits that this analysis was done on. Disaggregation of respondents whose gender was reported shows that both males and females at the experimental schools reported significant increase in frequency of hands-on activities compared to their counterparts at the control schools. The males reported an even higher increase than the females. Research Question 1 (Exam Scores) The results of the student scores on the old (February 2017) BECE exam administered before the intervention (Pre-Test) are shown in Figure 4 (left). The scores are presented in the form of box-and-whisker plots to capture the average (marked with an “X”), the range of the 1st to 3rd quartile (the box edges), and the overall range (the whiskers). One box-and- whisker plot is shown for each school, and the results are grouped into the three Circuits. The experimental schools are marked in red and the control schools are marked in blue. Before the intervention, all schools scored between 20 to 39%, on average. On average, the urban schools (Kwabenya Circuit: ES1, CS1) performed slightly higher than the rural schools before the intervention (35% vs. 27%). On average, there was minimal difference in the pre-test results between the experimental schools and their counterparts within their own Circuit. In each Circuit, the experimental schools scored slightly higher on average (Circuit 1 by 8%, Circuit 2 by 1%, Circuit 3 by 2%). Figure 4 (right) shows the student exam scores after the intervention. All schools improved on the exam, with post-test averages at each school ranging from 45 to 72%. The overall range of results increased for most of the schools, indicating a widening of the gap between the higher- and lower-performing students. The Ashalaja Circuit schools (ES2, ES3, CS2) were excluded from here on. Global Journal of Transformative Education (2020) Vol 2 Table 2. Significance of change in SQ3 (frequency of hands- on activities) results. Figure 4. School-level Pre-Test and Post-Test exam scores. Beem, Effect of Hands-on Science Activities 25 In order to determine the effect of the intervention beyond any learning that occurred in the normal mode of teaching (as observed in the control schools), the difference between the pre- and post-tests was calcu- lated for each school. These results are shown in Figure 5. Each experimental school(s) is best compared to the control school(s) within its own Circuit. In the Kwaben- ya Circuit, the experimental school (ES1) showed a larger improvement in learning than its control school (CS1): 33% vs. 20%. In the Kofi Kwei Circuit, one experimental school (ES4) showed a smaller improve- ment than the control schools (CS3, CS4) (9% vs. 25% average) while the second experimental school (ES5) showed a much bigger improvement (48%) than those control schools. The overall difference-in-differences of exam scores was +10.9%, with the experimental schools improving more than their counterparts over the course of the year. The difference-in-difference within Circuit 1 alone was +12.8%. Circuit 3 had mixed results across the different schools, resulting in an aver- age difference-in-differences +4.1% in that Circuit. Results from the surveys and follow-up visits reveal reasons for the difference in results with- in Circuit 3. Table 3 presents these results, disaggregated across gender. Note that gender information was missing for one student, so there is a discrepancy in sample size (N) between the overall value and the disaggregated ones. The female students improved 14.5% more than their counterparts, which is a greater change than the male students experienced (5.3% more than their counter- parts). Research Question 2 (Attitudes to Learning Science) Enjoyment of learning science (SQ 6-7) The results of the survey question that elicited stu- dent enjoyment of science (SQ6) are shown in Figures 6 and 7 (before and after the intervention, respectively). On average, students in both sets of schools reported similar attitudes before the intervention: 89% of ex- perimental school students responded that they ”Enjoy Science” as did 84% of the control school students. In the schools that did not receive PEN’s interven- tion, student interest decreased over the course of the 2.5 months. The opposite trend was recorded in the experimental schools. Those who reported enjoying sci- ence in the experimental schools increased by 7% and their counterparts in the control schools decreased by 15%. Therefore, on average, the students who received the intervention experienced a 22% greater increase in enjoyment of science. The open-ended responses to SQ7 reveal some reasons why students enjoyed learning science. Examples include “The teachers make it interesting,” “The practicals make it more fun,” “helps me to define things in my own words,” “helps me to picture,” “helps Global Journal of Transformative Education (2020) Vol 2 Figure 5. School-level change in exam scores over the course of the intervention. Table 3. Difference-in-differences of exam scores at experi- mental schools (ES) and control schools (CS). Figure 6. Pre-Test student survey responses on enjoyment of science at the experimental and control schools Beem, Effect of Hands-on Science Activities 26 give me an idea about the topic,” “helps in making learning easier,” “helps me remember well,” Engagement level,ease of learning science (SQ 4-5) A summary of the results of changes in student engagement level and ease of learning science are captured in Table 4. Both questions were answered on a scale of 1 to 7. The mean of the change in values (∆) between all the control schools and experimental schools is shown for each question. The significance of the difference between the two sets is captured through the t-test and Hedge’s g test. The sample size (N) of respondents (those from the 2 eligible Circuits and who answered both the pre and post surveys) is listed. For SQ4 “How engaged are you in the hands-on activities?”, a significant difference and large effect size were measured (p = 3x10-7, g = 0.85). This reveals that the intervention had a significant impact on the level of engagement that the experimental school students felt, compared to their counterparts. There was, however, no significant difference or effect size in the pre-post change between the control and experimental schools for SQ5 “How easy is it to learn science?” (p = 0.672, g = 0.07). Disaggregation of this data across gender and geography is captured in Table 5. Both genders experienced a change with large effect size for SQ4 (Engagement), but the effect size on the females was even higher, nearly reaching a value of 1 (gfemale = 0.92, gmale = 0.77). Disaggregation of this data across urban schools (Kwabenya Circuit) and rural schools (Kofi Kwei Circuit) also reveals two differences of note. For SQ4 (Engagement), there was a large effect size for the urban schools (g = 0.981) compared Global Journal of Transformative Education (2020) Vol 2 Figure 7. Post-Test student survey responses on enjoyment of sci- ence at the experimental and control schools Table 4. Significance of change in SQ4-5 (level of engagement, ease of learning science) results Table 5. Significance of change in SQ4-SQ5 results, disaggregated across gender and geography to a medium effect size for the rural schools (g = 0.528). One of the experimental schools in the rural circuit (ES4) only reported a change of 1.33 points out of 7, on average, on this survey questions. The other experimental schools (ES5 and ES1) had changes of 2.143 and 2.204 points out of 7, respectively. Note that ES4 is the same school whose students experienced less improvement on their exam scores than its counterparts (Figure 5), suggesting that this school may not have implemented the intervention as effectively as the other two experimental schools. For SQ5 (Ease), the urban schools experienced no effect size (gurban = 0.059) but the rural schools did experience a small effect size (grural = 0.284). Beem, Effect of Hands-on Science Activities 27 Research Question 3 (Interest in STEM Majors and Careers) Results from the final two survey questions are shown in Table 6 and are disaggregated across gen- der. When asked of the SHS major they intended to study and the career they were interested to pursue, the males at the experimental schools did not shift their response between non-STEM and STEM options with statistical significance or effect size (g = 0.07), compared to their counterparts at the control schools. The females at the experimental schools, however, did significantly shift towards both. They shifted towards STEM majors with small effect size (g = 0.22) and to- wards STEM careers with medium effect size (g = 0.48), compared to their counterparts at the control schools. Follow-up visits to the schools Remarks by the Trainers and Teachers were cap- tured during follow-up visits to each of the sites, except for the schools in Ashalaja Circuit, due to an inability to establish communication with the Trainer. They cited factors that influence the state of the learning environment in each respective school. These are captured in Table 7 along with observations made by the data collection assistant, which are categorized as that of the Researcher. From these remarks and observations, it is seen that the pair of schools in the Kwabenya Circuit had comparable key factors in terms of their learning environment. The infrastructure in both schools was similar and both teachers had similar levels of teaching experience. They also did not experience any major attendance issues. Hence, in this pair of schools, the intervention was able to be conducted as intended, and with an accurate comparison. For the Kofi Kwei Circuit, the Trainer observed that learning gains in his Circuit, which is a rural one, would be harder to come by than those achieved in the urban schools. Visits to the two experimental schools in his Circuit revealed key differences between them. ES4 struggled to get regular attendance, due to the school’s poor infrastructure. The infrastructure was of such low quality that the school would not operate during any day in which it rained. Also, the students there were Global Journal of Transformative Education (2020) Vol 2 Table 6. Significance of change in SQ1-2 (desired SHS major, career) results, disaggregated across gender Table 7. Remarks and observations captured during follow-up visits to the schools found to have poor English reading ability. Finally, the teacher being trained through the intervention had been placed at the school through Ghana’s mandatory National Service program, meaning that he was a fresh universi- ty graduate, with no prior teaching experience. For these reasons, the ES4 students did receive the intended dosage of the intervention. Evidence collected in the follow-up visit to ES5 revealed that the intervention’s implementation had not only been consistent, but eagerly adopted by the school’s Teacher. He appeared to have fallen in love with the science con- tent, as he had gone the extra mile to create science-based Beem, Effect of Hands-on Science Activities 28 posters to hang around the class. This was not an explicit suggestion in the training provided. The school infrastructure was also of sufficient quality to allow constant attendance during poor weather conditions. The Trainers and Teachers also made suggestions for improving the intervention next time, and these remarks have been captured in Table 8. All agreed that this intervention was helping their students, but they suggested that if further learning gains are desired, then the approach should be intensified. They sug- gested commencing the intervention with students in earlier year groups, increasing the frequency of the intervention, and increasing the number of resources used, so as to enable smaller group sizes. Discussion Over the course of a short (2.5 month) period, this intervention enabled significant improvements in Ghanaian students’ science education experience, as a result of the introduction of hands-on activities into the classroom. Key quantitative results mentioned in the previous section are combined with the qualitative perspectives elicited in the follow-up visits for discus- sion here. Research Question 1 (Exam Scores) Overall, the intervention improved the exam scores for students at the experimental schools 10.9% more than the control schools over the 2.5-month period. The comparison between schools was the cleanest at the Kwabenya Circuit, where both schools are situated in the same cluster and have similar teacher quality. The difference-in-difference of exam scores there was 12.8%. One experimental school (ES4) did not improve as much as its counterparts, but this can be attributed to their inconsistent implementation of the intervention. The experimental school who recorded the greatest im- provement overall (ES5) found the program to be such a significant benefit that their headteacher made a visit to the District Education Office specifically to thank them for supporting this intervention. Directly after this study was completed, they also announced that they had achieved the highest score ever documented in the school’s history on the science portion of the BECE that year. The female students across all schools improved their exam scores more than their male counterparts (14.5% vs. 5.3%), revealing the significant finding that this hands-on teaching approach can disproportionately enable learning gains for females. Given the widespread theme of males scoring higher than females in science/STEM subjects, this result merits further investigation. Few approaches in Sub-Saharan Africa have been measured to enable significant improvement in exam scores in STEM fields, much less to disproportionately favor learning gains for females. It is also worth noting that the females reported less increase in frequency of hands-on activities experienced in their classrooms than their male colleagues. Nonetheless, the females managed to improve their learning outcomes more than the males. The reported lower frequency could be a result of teachers engaging the males more than females in the execution of the activities and/or the male students dominating the activities. These phenomena have been documented in Western literature (Jovanovic & King, 1998; Tobin & Garnett, 1987). Further studies in Ghana should include a specific gender lens in the lesson observation to determine the gender dynamics at play in hands-on activity execution. Research Question 2 (Attitudes to Learning Science) Another substantial result is that while student enjoyment of science decreased (−15%) in the control schools over the course of the intervention, there was an increase (+7%) in enjoyment of science in the Global Journal of Transformative Education (2020) Vol 2 Table 8. Remarks captured during follow-up visits to the schools: suggestions for improving the intervention’s efficacy Beem, Effect of Hands-on Science Activities 29 experimental schools. The inclusion of hands-on activities in the classroom countered the natural loss of enjoyment in science that students experienced in standard classrooms. Gains in interest in science/STEM likely play a role in enabling gains in the aforemen- tioned exam scores. The level of engagement that the experimental school students reported had a significant increase with large effect size compared to their counterparts. The females reported an even larger increase than the males. This indicates once again that the female stu- dents in this study were even more positively affected by this intervention than the males. If the previous explanation for the females’ response to “frequency of activities” is accurate, then female students could stand to benefit from even higher engagement level increases if teachers are guided to involve female students more in the activities. There was no significant improvement in the ease of learning science, either for males or females. Longer interventions should be conducted to determine if the gains in enjoyment and engagement captured here can translate into deeper learning of and comfort with science concepts. Research Question 3 (Interest in STEM Majors and Careers) Interestingly, although student enjoyment of sci- ence improved at the experimental schools, a univer- sally equivalent shift towards selecting a Science-based program to study in SHS was not measured. The female students did positively shift with a small effect size (gfemale = 0.22), but their male counterparts did not (gmale = 0.07). One teacher in the study offered an interpre- tation of this: “[The students] think they might not get the required grades to get admission for the Science... They have the zeal, but academically they are not good enough to be taken for Science...The system tries to inhibit them.” These comments suggest that the Ghana- ian educational system is not structured to support all students with a growing interest in science. The admis- sion requirements to gain entry into a Science program are high, so a student may shut his or her mind to that as a viable option to pursue. It is also interesting to note that females did significantly shift to selecting STEM- based careers over the course of the intervention (p = 0.04, g = 0.48). The intervention again affected female students disproportionately, this time by opening up their mind to different aspirations for their long-term careers. Limitations Implementation Levels A few inconsistencies in levels of the intervention’s implementation across the experimental schools have been identified. These are summarized in Table 9. At the Kwabenya school (ES1), the Trainer and Teacher implemented the intervention thoroughly, as evidenced by the high level of change reported by students in their level of engagement in the lessons (Table 5, SQ4, Urban). Follow-up visit observations confirm that the students regularly attended and therefore had the intended dosage of exposure to the intervention. The implementation at the Ashalaja schools (ES2, ES3) had begun, but was incomplete, due to the trainer falling ill partway through. Finally, the Kofi Kwei schools (ES4, ES5) had different levels of implementation. The ES4 Teacher did not completely implement the intervention, as it was revealed during the follow up visit that students had not regularly attended school during the intervention period. ES5, on the other hand, had no attendance issues. Their teacher not only implemented the intended intervention, but he appeared to amplify his craft as a result. These levels of implementation quality have been kept in focus for interpretation and discussion of the results. Future studies must put stronger measures in place to ensure thorough implementation in all experimental schools. School Selection Process The school selection process employed did not compare exam results explicitly, but the trainers were tasked to carry out the school selection based on their knowledge of the schools’ general performance. Future studies should include a more thorough list of criteria for school selection. This will help both to ensure the schools’ ability to implement the intervention and it will also enable a clearer comparison between the pair of experimental and control schools. Global Journal of Transformative Education (2020) Vol 2 Table 9. Assessment of implementation level at each experimental school Beem, Effect of Hands-on Science Activities 30 Programmatic & Policy Implications A few potential implications for PEN’s program- ming and other African teacher training organizations arise from these results. Given the strong correlation between implementation level and learning gains achieved in this study, PEN should develop systems to ensure that the methodologies their teachers are trained on get implemented in the classroom. This will involve working even more closely with key stakehold- ers, such as District Education Officers, headteachers, and national education bodies to ensure that monitor- ing activities reinforce the use of hands-on activities and that curriculum standards also encourage teachers to use hands-on activities. Given the example of the E5 school achieving its historically highest national exam score in science after going through this intervention, and thanks to the teacher there being enthusiastic enough to take up the new approach, PEN should consider identifying motivated teachers and focusing their interventions on them. A focus on intrinsically motivated teachers will likely result in higher learning gains than would be achieved through its current approach of offering train- ing to all available participants. This must, of course, be balanced with the organization’s goal of seeing wide- scale adoption of hands-on pedagogies. PEN and other African STEM organizations should consider pursuing a specific gender focus to their pro- gramming, since the gains achieved among the female students are strong in both attitudes and learning out- comes . Furthermore, PEN and other teacher training organizations should consider including a component of its training that guides teachers to reduce any bias they may have in involving male students more than females in the activities. Ghana’s government has a goal of seeing 60% of its university students pursuing STEM majors. A strong STEM pipeline will be achieved by having students first gaining interest in the subject, then electing to study STEM subjects, and then pursuing STEM careers. This short intervention achieved strong gains in building in- terest and small gains in shifting future career pursuits. Future studies should be carried out to determine the length and type of intervention needed to achieve large gains in shifting career pursuits towards STEM disci- plines. Policymakers should consider the admission requirements and societal perceptions around pursuing Science majors/disciplines as they formulate plans to achieve their goal. Conclusion A quasi-controlled experimental study was conducted in 2017 with 9 schools in Greater Accra Region to preliminarily assess the impact on Ghanaian students when their teachers are trained to carry out hands-on activities with locally-available materials. On average, students undergoing the intervention improved their exam scores by 10.9% more than their counterparts over the course of the 2.5-month period, but discrepancies existed at the school level. These discrepancies are attributable to challenges that impeded a complete implementation of the intervention at select schools. In spite of implementation challenges, the students in the experimental schools still experienced significant shifts in their attitudes and interests over the course of the study. Students undergoing the intervention increased in their enjoyment of science 22% more than their counterparts, and their levels of engagement increased significantly (p = 3 x 10-7, g = 0.85). Female students experienced greater gains than their male counterparts in terms of exam scores, engagement levels, and interest in pursuing STEM majors and careers. And despite the commonly held belief that learning gains are slower in rural schools than urban ones, the school with the highest exam score gains in this study was in a rural area. They even broke their own school record for national exam scores immediately following this intervention. A longer intervention is likely needed to achieve significant gains in student response to the ease of learning science. Preliminary evidence for multiple lines of questioning around the role of hands-on activities in the Ghanaian context have been brought to the fore in this study. The findings add to a nascent body of knowledge of the effect of hands-on pedagogies in the West African context. The effect of experiential STEM pedagogies on student attitudes, career interest, and societal barriers have been uncovered, call for deeper investigation, and merit attention from policymakers. A multitude of factors influence the efficacy of interventions such as this one. In order to accurately assess the role of practical teaching methodologies in the Ghanaian classroom, a greater number of these factors should be considered in the school selection process. In particular, the level of infrastructure should meet a certain level in order for the school to be selected. If poor infrastructure prevents the students from coming to school, it does not matter whether their teacher is equipped to teach with practical activities Global Journal of Transformative Education (2020) Vol 2 Beem, Effect of Hands-on Science Activities 31 Global Journal of Transformative Education (2020) Vol 2 that week or not. A refined version of this study was undertaken in the 2017/8 academic year (Babb & Stockero, 2020) to better control for some of these variables. Geographic proximity is no longer being considered as sufficient evidence of school similarity. School infrastructure, teacher background, and students’ prior years exam performance are also now being considered. Successful findings must be scaled up nationwide to shift the status quo for STEM teaching and learning in Ghana. Acknowledgements The author would like to thank Evans Sackitey, Peace Sampson, and Fredrick Ohene Atta Opoku for their contributions to the logistics of this study. Thanks to Jacob Babb for his input in ideating some of the initial designs of this study. Dr. Anne Marshall of MIT’s Teaching and Learning Lab was a key resource in the design of the study as well. Finally, thanks to the PEN trainers, teachers, headteachers, Ghana Education Service officials, and students who enabled this study to be completed. References Abdi, A. (2014). The effect of inquiry-based learning method on students’ academic achievement in science course. Universal Journal of Educational Research, 2(1), 37–41. Aboagye, G. K. (2009). Comparison of learning cycle and traditional teaching approaches on students’ understanding of selected concepts in electricity. Doctoral dissertation, University of Cape Coast. Abrahams, I., & Millar, R. (2008). Does practical work work? A study of the effectiveness of practical work as a teaching and learning method in school science. International Journal of Science Education, 30(14), 1945–1969. Adomako Ampofo, A. & Kaufmann, E.E. (2018). Interview by K. Simtim-Misa. KSM Show- Prof Akosua Adomako Ampofo and Dr. Elsie Effah Kaufmann, Hanging out with KSM part 2. [YouTube Clip]. Aizenman, N. & Warner, G. (2018). What can we learn from Ghana’s obsession with preschool? National Public Radio. Babb, J. J., & Stockero, S. L. (2020). Impact of Practical Education Network on students in selected Ghanaian junior high school science classrooms. African Journal of Research in Mathematics, Science and Technology Education, 1-13. Bando, R., Näslund-Hadley, E., & Gertler, P. (2019). Effect of inquiry and Problem Based pedagogy on learning: Evidence from 10 field experiments in four Countries. National Bureau of Economic Research (NBER) Working Paper No. 26280. https://files.eric.ed.gov/fulltext/EJ1053967.pdf https://files.eric.ed.gov/fulltext/EJ1053967.pdf https://files.eric.ed.gov/fulltext/EJ1053967.pdf https://www.tandfonline.com/doi/abs/10.1080/09500690701749305 https://www.tandfonline.com/doi/abs/10.1080/09500690701749305 https://www.tandfonline.com/doi/abs/10.1080/09500690701749305 https://www.tandfonline.com/doi/abs/10.1080/09500690701749305 https://www.youtube.com/watch?v=I_lXmwfORQM https://www.youtube.com/watch?v=I_lXmwfORQM https://www.youtube.com/watch?v=I_lXmwfORQM https://www.npr.org/sections/goatsandsoda/2018/06/19/613655788/when-it-comes-topreschool-does-father-really-know-best https://www.npr.org/sections/goatsandsoda/2018/06/19/613655788/when-it-comes-topreschool-does-father-really-know-best https://www.tandfonline.com/doi/abs/10.1080/18117295.2020.1814662 https://www.tandfonline.com/doi/abs/10.1080/18117295.2020.1814662 https://www.tandfonline.com/doi/abs/10.1080/18117295.2020.1814662 https://www.nber.org/papers/w26280 https://www.nber.org/papers/w26280 https://www.nber.org/papers/w26280 Beem, Effect of Hands-on Science Activities 32 Blench, R., & Dendo, M. (2006). A dictionary of Ghanaian English. Buabeng, I., Ossei-Anto, T. A., & Ampiah, J. G. (2014). An investigation into physics teaching in senior high schools. World Journal of Education, 4(5), 40–50. Buabeng, I., Owusu, K. A., & Ntow, F. (2014). TIMSS 2011 science assessment results: A review of Ghana’s performance. Journal of Curriculum and Teaching, 3(2), 1-12. Davis, E. K., & Chaiklin, S. (2015). A radical-local approach to bringing cultural practices into mathematics teaching in Ghanaian primary schools, exemplified in the case of measurement. African Journal of Educational Studies in Mathematics and Sciences, 11, 1-15. Donkor, A. K., & Justice, D. K. (2016). Girls education in science: The challenges in northern Ghana. Journal of Education & Social Policy, 3(1), pp 82-96. Ergül, R., Şımşeklı, Y., Çaliş, S., Özdılek, Z., Göçmençelebı, Ş., & Şanli, M. . (2011). The effects of inquiry-based science teaching on elementary school students’ science process skills and science attitudes. Bulgarian Journal of Science and Education Policy (BJSEP), 5(1), 48–68. Gormally, C., Brickman, P., Hallar, B., & Armstrong, N. (2009). Effects of inquiry-based learning on students’ science literacy skills and confidence. International Journal for the Scholarship of Teaching and Learning, 3(2). 1-22. Haffar, A. (2016). State of Ghana’s educational system destroys creativity? Graphic Online. Johnston, J., & Ksoll, C. (2017). Effectiveness of Interactive Satellite-Transmitted Instruction: Experimental Evidence from Ghanaian Primary Schools. Stanford Center for Education Policy Analysis (CEPA) Working Paper No. 17-08. Jovanovic, J., & King, S. S. (1998). Boys and girls in the performance-based science classroom: Who’s doing the performing? American Educational Research Journal, 35(3), 477-496. Kibirige, I., Rebecca, M. M., & Mavhunga, F. (2014). Effect of Practical Work on Grade 10 Learners’ Performance in Science in Mankweng Circuit, South Africa. Mediterranean Journal of Social Sciences, 5(23), 1568–1577. Ministry of Education Ghana. (2019). Science Curriculum for Primary Schools. Accra, Ghana: National Council for Curriculum and Assessment, Ministry of Education Ghana. National Council for Curriculum & Assessment. (2019). Teacher Resource Pack – Primary Schools. Accra, Ghana: Ministry of Education Ghana. Practical Education Network. (2016). Field Research Report on STEM Teacher Training Project. PEN. Practical Education Network. (2020). Our Work. PEN. Quansah, F., & Asamoah, D. (2019). Chew, Pour, Pass, and Forget: Students perception of authentic assessment in universities in Ghana. Social Science and Humanities Journal, 3(3), 901-909. Tobin, K., & Garnett, P. (1987). Gender related differences in science activities. Science Education, 71(1), 91-103. van den Hurk, A., Meelissen, M., & van Langen, A. (2019). Interventions in education to prevent STEM pipeline leakage. International Journal of Science Education, 41(2), 150-164. World Economic Forum. (2017). The future of jobs and skills in Africa preparing the region for the fourth industrial revolution. Executive Briefing. Cologny/ Geneva, Switzerland: World Economic Forum. Heather Beem, Ph.D., (hbeem@ashesi.edu.gh) is an Engineering Lecturer at Ashesi University, Berekuso, Ghana. She is also the CEO and Founder of the Practical Education Network. Author http://www.rogerblench.info/Language/English/Ghana%20English%20dictionary.pdf http://www.rogerblench.info/Language/English/Ghana%20English%20dictionary.pdf https://files.eric.ed.gov/fulltext/EJ1158569.pdf https://files.eric.ed.gov/fulltext/EJ1158569.pdf https://files.eric.ed.gov/fulltext/EJ1158569.pdf https://files.eric.ed.gov/fulltext/EJ1157678.pdf https://files.eric.ed.gov/fulltext/EJ1157678.pdf https://files.eric.ed.gov/fulltext/EJ1157678.pdf https://www.ajol.info/index.php/ajesms/article/view/168994/158457 https://www.ajol.info/index.php/ajesms/article/view/168994/158457 https://www.ajol.info/index.php/ajesms/article/view/168994/158457 https://www.ajol.info/index.php/ajesms/article/view/168994/158457 http://jespnet.com/journals/Vol_3_No_1_March_2016/10.pdf http://jespnet.com/journals/Vol_3_No_1_March_2016/10.pdf http://see-articles.ceon.rs/data/pdf/1313-1958/2011/1313-19581101048E.pdf http://see-articles.ceon.rs/data/pdf/1313-1958/2011/1313-19581101048E.pdf http://see-articles.ceon.rs/data/pdf/1313-1958/2011/1313-19581101048E.pdf https://files.eric.ed.gov/fulltext/EJ1136707.pdf https://files.eric.ed.gov/fulltext/EJ1136707.pdf https://www.graphic.com.gh/business/business-news/state-of-ghana-s-educational-system-destroys-creativity.html https://www.graphic.com.gh/business/business-news/state-of-ghana-s-educational-system-destroys-creativity.html https://www.mcser.org/journal/index.php/mjss/article/viewFile/4693/4553 https://www.mcser.org/journal/index.php/mjss/article/viewFile/4693/4553 https://www.mcser.org/journal/index.php/mjss/article/viewFile/4693/4553 https://www.mcser.org/journal/index.php/mjss/article/viewFile/4693/4553 https://nacca.gov.gh/wp-content/uploads/2019/06/TEACHER-RESOURCE-PACK-P1-P6.pdf https://www.practicaleducationnetwork.com/our-work http://sshj.in/index.php/sshj/article/download/309/135 http://sshj.in/index.php/sshj/article/download/309/135 http://sshj.in/index.php/sshj/article/download/309/135 https://www.researchgate.net/profile/Pamela_Garnett2/publication/229797557_Gender_differences_in_science_activities/links/5a1a436c0f7e9be37f9a60e4/Gender-differences-in-science-activities.pdf https://www.researchgate.net/profile/Pamela_Garnett2/publication/229797557_Gender_differences_in_science_activities/links/5a1a436c0f7e9be37f9a60e4/Gender-differences-in-science-activities.pdf https://www.tandfonline.com/doi/pdf/10.1080/09500693.2018.1540897 https://www.tandfonline.com/doi/pdf/10.1080/09500693.2018.1540897