































ii 

Annals of Applied Psychophysiology December 2024 Volume 11 
______________________________________________________________________________ 

 
Assessing the Effectiveness of Training Logic and Reasoning Skills 
Among Middle and High School Students Using a Retrospective Pre 
to Post Training Assessment  
Gabrielle	D.	Timlin1	
Published online: December 2024 

  © The Author(s) 2024 

Abstract 
Background: Both students studying psychophysiology and professionals performing research in 
the field need to be versed in the basics of logic so they can do a better job assessing the literature 
and designing their studies. Hence, it is imperative to assist middle and high school students with 
the development of this crucial skill. The purpose of this study was to assess the results obtained 
from instruction in logic in middle and high school students. This retrospective study ascertained 
if students gained improved logic and reasoning skills after participating in the specified course in 
formal logic. 
Methodology: Logic and critical thinking skills were taught using an established curricula with a 
custom pre and post assessment to determine baseline and learning. The participants were middle 
and high school students for whom one-half hour of instruction for ten days was conducted in a 
regular classroom setting, as part of their regular instruction. The first and last days consisted of 
pre and post-assessments using custom pre and post-assessments of their logical abilities. The 
instruction on days 2-9 was accompanied by clearly identified learning goals, follow-up group 
Socratic discussion, and written individual exit tickets. Data collection consisted of documenting 
assessment results. 31 matched pairs for Group 1 (the middle school students) and 32 matched 
pairs for Group 2 (the high school students) were analyzed using the Wilcoxon Signed Rank Sum 
test (Glanz, 2012).   
Results: Overall, students performed better on the post-test after instruction. The data from the 
assessments were gathered retrospectively for the purposes of this study.  Results indicate that this 
curriculum is a good choice for logic instruction for middle and high school students. 
Conclusion: Future research should include larger sample sizes, student adherence to instruction 
and participation, inclusion of the psychomotor learning domain and validity. 
 
 
 
Keywords: Psychophysiology, Students, Curriculum, Logic and Critical Thinking Skills 

1. Gabrielle D. Timlin1 
Saybrook University, Pasadena, California, USA 

 
 
 
 
 



iii 

Introduction 
 

Both students studying psychophysiology and professionals performing research in the 
field need to be versed in the basics of logic so they can do a better job assessing the literature 
and designing their studies. Genesereth and Chaudhri (n.d.) assert that teaching logic is 
imperative for the creation of a responsible population that is equipped to detect logical fallacies, 
engage in good decision making, think critically and question their appointed leaders with skill. 
Students of mathematics benefit from learning logic when attempting to solve difficult problems 
like proofs (Bako, 2002). However, people who are neither mathematicians nor scientists may 
not continue to use these disciplines as an analytic foundation (Kobylarek, 2020). Furthermore, 
the exercise of analyzing one’s own thinking through the Socratic method elucidates gaps in 
thinking using questions to pursue logic and reasoning (Perdue, 2014). A solution is to teach 
formal logic as it is applicable to daily life (Genesereth & Chaudhri, n.d.). Modern researchers 
have provided updated guidelines and structure for teaching and assessing logic and critical 
thinking, including Robinson (2011) who argues that logic education should be relatable to key 
studies.  

The purpose of this retrospective project was to explore the results obtained from a short-
term implementation of a course of study designed to develop skills in logic and critical thinking 
in middle and high school students. The research question to be answered in this study was, do 
students gain improved logic and reasoning skills after participating in this course in formal 
logic? 

 
Methodology 

The logic training program was performed in two different school settings in classrooms 
as part of the regular student curriculum. 
Participants 

Classes of middle school and high school students received pre- and post-tests with logic 
instruction in between. The researcher moved from one school to another and taught the same 
logic module in each school. The first group consisted of 31 total matched sets of 8-12 grade 
students and the second group consisted of 32 matched sets of 12 grade students only, where 
matched sets refer to pre and post-tests. The first group had 2 classes of 65 total 8th grade 
science students, 2 classes of 9th -12th grade science students of 27 total, and 1 class of 5 total 
physics senior high school students and one junior. Of these, the 8th grade classes had 25 males 
and 40 females, the Earth Science classes had 16 males and 11 females, and the physics class had 
3 males and 2 females. The second group included 4 classes of 74 seniors total comprised of 31 
males and 43 females. All groups were de-identified. Both groups were comprised of students 
from middle-class socioeconomic backgrounds. Out of both groups, there were students who 
elected not to participate. Additionally, the instructions were to choose a non-identifiable name 
consisting of a letter, a number and a symbol and use that identifier for both pre and post-tests. 
As not everyone chose to follow the instructions and tests were matched afterward, 31 matched 
sets from group 1 were used and 32 matched sets from group 2 were used as there was strong 
confidence that these were indeed matched sets.  

Both schools were the workplace of the researcher who moved from one place to the 
next. The researcher both taught and assessed classes in logical thinking using a questionnaire 



iv 

created for that purpose. The research was limited by the retrospective nature of that intervention 
and data. This was a retrospective study that utilized data collected from a middle school and a 
high school which had modules emphasizing instruction in logic in place. As part of the 
curriculum, the instructor developed pre and post-training questionnaires to assess the 
effectiveness of the instruction.  

Each school gave written permission to use the retrospective data gathered while teaching 
the logic modules.  Both the instrument and the instruction were all given in English. All 
students participated in the units as these were required, however students were given the option 
to refrain from participating in completing the inventories. Only 3 out of the 100 students in 
Group 1 (middle school) and 7 out of the 74 students in Group 2 (high school) elected to not 
participate in completing the inventories. 

 
Procedure 

A baseline was obtained in the form of an assessment used to determine current ability in 
logic, reasoning, and critical thinking. The Inventory of Instruments of Critical Thinking was 
consulted (Follman, et al., 1996). Several tests were ruled out as they were either inappropriate 
for the intended age group, older, test-retest reliability was poor, such as free online tests 
available at the time, or student laptops were not consistently available. Additionally, while the 
Test of Logical Thinking (TOLT) exists for 6th grade through college, this instrument assessed 
information already being assessed in math and science classes at the sites (Tobin & Capie, 
1981). It did not assess whether logic skills acquired in math and/or science resulted in increased 
student ability to reason in logical fallacies through conversation. After reviewing the literature, 
no short instrument was discovered that was suitable to the intended population or purpose. 
Hence, custom pre and post-tests based on logical fallacies were created (Sherman, 2021). The 
questions with logical fallacies are shown below in Figures 1-2. 

It is a standard procedure for inventories given close together in time to have different 
versions and/or test questions that test the same information; hence it was necessary for the pre 
and post-tests to have different questions. The questions were initially informally evaluated for 
validity and understanding by two teachers and ten students, five of whom were girls, and five of 
whom were boys, at the first school where the instruction was to be tested. No concerns were 
registered by any student or teacher. 

Prior to participating, the learning outcomes were clearly stated. After participating, 
students were required to produce outcomes, which were a variety of spoken group discussions 
or written feedback. Because the baseline assessment was likely to provide valuable information 
from which lessons can be best determined, because lessons were taken from an established 
curriculum, and because this was integrated into different science classes, structured lesson plans 
were not included in this protocol (Cothran, 2012; Cothran, 2017; Cothran 2018a; Cothran 
2018b).   

After the initial assessment on Day 1, lessons were given on Days 2-9 during regular 
class time, with stated desired outcomes, activity, Socratic discussion with questions intended to 
elicit higher thought, and a written exit ticket from each student on the topic of the day’s learning 
objectives. Questions may be an analysis of a logical fallacy, for example. These lessons 
consisted of lessons from the Traditional Logic 1 curriculum that were designed to enhance logic 
and critical thinking skills, but that were not correlated to the test (Cothran, 2012; Cothran, 2017; 
Cothran 2018a; Cothran 2018b). Since the content was correlated to the test, this reduced the 
likelihood that the students were trained to succeed on an assessment as opposed to reaching the 



v 

desired goal: strengthening their logic and critical reasoning skills. This curriculum utilizes a 
student workbook and textbook. A mixture of individual tasks, small group work and whole class 
readings and discussions were utilized. The assessment was given again on Day 10 to ascertain 
the extent to which students learned logic and critical thinking skills. Because it was unlikely that 
a perfectly correct or a perfectly incorrect answer would be given in all cases, a Likert scale was 
created to grade the pre and post assessments.  

 
Results 

In most cases, the p value was less than the significance level (alpha) meaning that the 
results are statistically significant and there is a difference between the pre and posttests, for a 
two tailed (non-directional) interpretation, which means that students either improved or declined 
in skill (Glanz, 2012). The exception to this is the third question for Group 1 where there is no 
difference between the pre and posttests. However, a one-tailed interpretation is necessary 
because the point of the intervention is to determine if students are able to employ logic better 
post instruction than before. To do this, the significance level needs to be divided by two. This 
means that 0.05 which indicates a 95% level of confidence, needs to be divided by two to 
become 0.025.  Hence, when using total averages in the Wilcoxon Signed Rank Sum, each group 
showed improvement post instruction (Glanz, 2012). When averaging each individual question, 
most questions indicated improvement post instruction except for Questions 1 and 4 in Group 2, 
and Question 3 in Group 1, where no improvement was indicated.  

 In each case, the W score indicates that many participants had the same rank, except for 
Question 1 for Group 2 (Glanz, 2012). The second question yielded the highest W score for each 
group, thus skewing the total average for the W score to a higher number than it would have 
otherwise been. In comparison, question 3 yielded the highest P score for group 1 and was the 
only question for Group 1 where there was no positive difference between the intervention of 
logic education and no intervention at all. This was the case for questions 1 and 4 for Group 2. A 
Wilcoxon Signed-Rank Test indicated that scores were significantly different before and after the 
intervention when averaged for all questions, with W= 373, p = 0 for Group 1 and W = 310, p = 
0 for Group 2 (Glanz, 2012). For each independent question, the only question Group 1 did not 
show improvement with the intervention was question 3. For Group 2, it was questions 1 and 4. 
 
 
 
 
 
 
 
 
 
 
 
 
 
 



vi 

 

Table 1 

Student Breakdown. 

  Group 1 Group 2 
# of students 
who completed 
the 
questionnaire as 
a matched set 31 32 
8th Grade 
Science total 
students 65 0 
8th grade 65 0 
Males 25 0 
Females 40 0 
Earth Science 
total students 27 0 
Males 16 0 
Females 11 0 
9th grade 21 0 
Males 12 0 
Females 9 0 
10th grade 3 0 
Males 2 0 
Females 1 0 
11th grade 2 0 
Males 2 0 
Females 0 0 
12th grade 6 74 
Males 3 31 
Females 3 43 
Physics total 
students 5 74 

 

 

 

 



vii 

Table 2 

Statistics By Group (Glanz, 2012). 

Test Group 1 Group 2 
Standard Deviation 0.82 0.49 
Power Analysis 1 0.989 
Wilcoxon Avg W 373 310 
Wilcoxon Avg P 0 0 
Wilcoxon Q1 W 120 0 
Wilcoxon Q1 P 0 >0.046 
Wilcoxon Q2 W 337 435 
Wilcoxon Q2 P 0 0 
Wilcoxon Q3 W 107 88 
Wilcoxon Q3 P 0.057 0.007 
Wilcoxon Q4 W 182 61 
Wilcoxon Q4 P 0 0.044 
Wilcoxon Q5 W 135 72 
Wilcoxon Q5 P 0.003 0.019 

 

Figure 1 

Box and Whisker Plot for Group 1 W Values Distribution by Question Averages 

W
 V

al
ue

s

0

50

100

150

200

250

300

350

Group 1 W values by question

Group 1



ii 

Figure 2 

Box and Whisker Plot for Group 2 W Values Distribution by Question Averages 

 

Figure 3 

Box and Whisker Plot for Group 1 P Values Distribution by Question Averages  

The following are the data. 

 

W
 V

al
ue

s

0

50

100

150

200

250

300

350

400

450

500

Group 2 W values by question

Group 2

1

P 
Va

lu
es

0

0.01

0.02

0.03

0.04

0.05

0.06

Group 1 P Values by Question 

Group 1



iii 

Figure 4 

Box and Whisker Plot for Group 2 P Values Distribution by Question Averages  

 
 
 

 
 

Discussion 
With the exception of one question in Group 1 and two questions in Group 2, there is a 

statistically significant difference between the pre and posttests, indicating that this particular 
curriculum is a good choice for future logic instruction in middle and high school settings. Both 
sites were public classical charter schools. Since the population represented those students 
enrolled in specified science classes at that time, this sample population is representative of 
middle and high school students enrolled in classical charter schools. Hence, these results are 
generalizable to middle and high school students enrolled in classical charter schools. 

Limitations include small sample sizes in both groups, lack of student adherence to 
instructions, lack of student participation, short study duration, limited number of science classes 
and schools, lack of time to devote solely to logic in a science classroom and integration of logic 
into the science curriculum. Further limitations include a general reluctance by students to value 
logic instruction in a science classroom and answer interpretation done by only one person, the 
researcher. Finally, the exploratory, simple nature of the study and the survey are limitations. 
Delimitations involve teaching this in a science classroom. The activity was integrated into the 
science curriculum for middle and high school students in two schools. 

Recommendations include repeating instruction in a dedicated logical class including at 
least 30 minutes of class time up to one hour and consisting of a combination of individual 
reading to start class, teacher instruction and a question-and-answer session afterwards followed 
by small group participation where the students perform exercises in the curriculum and finally 
ending with an exit ticket demonstrating understanding of the day’s learning. Also, logic 

P 
Va

lu
es

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

0.045

0.05

Group 2 P Values by Question

Group 2



iv 

instruction should be connected with math and science classes reinforcing math and science 
instruction as well as be performed at sites that exemplify excellent teacher support and prioritize 
student responsibility for their own learning. Additionally, more research is necessary to study 
the inclusion of the psychomotor learning domain, as this learning domain is applicable in 
practical application, as opposed to simply acquiring information as in the learning domain. 
Furthermore, as this was exploratory research, evaluation of validity should be included in future 
research. 

Conclusion 

In conclusion, it is unwise to assume that students will learn logic and reasoning skills by 
exposure to math and science courses. Instead, logic and reasoning should be facilitated in 
middle and high school classes in conjunction with math and science education to facilitate grasp 
of basic concepts and promote student responsibility for independent thinking.



1 

References 
 

Aksu, G., & Koruklu, N. (2015). Determination the effects of vocational high school students’ 
logical and critical thinking skills on mathematics success. Eurasian Journal of 
Educational Research, 15(59). https://doi.org/10.14689/ejer.2015.59.11 

 
Bako, M. (2002). Why we need to teach logic and how can we teach it? International Journal for 

Mathematics Teaching and Learning, 1473. https://cimt.org.uk/journal/bakom.pdf 
 
Cothran, M. (2012). Traditional logic 1: Introduction to formal logic: Quizzes & tests (2nd ed.). 

Memoria Press. 
 
Cothran, M. (2017). Traditional logic 1: Introduction to formal logic: Student text (3rd ed.). 

Memoria Press. 
 
Cothran, M. (2018a). Traditional logic 1: Introduction to formal logic: Teacher key (3rd ed.). 

Memoria Press. 
 
Cothran, M. (2018b). Traditional Logic I: Introduction to formal logic: Workbook (Classical 

Trivium Core) (3rd ed.). Memoria Press. 
 
Crockett, L. (2023, July 26). 5 of the best practices for improving critical thinking skills: Future 

focused learning. Future Focused Learning. 
https://futurefocusedlearning.net/blog/critical-thinking/5-of-the-best-practices-for-
improving-critical-thinking-skills 

 
Follman, J., Lavely, C., & Berger, N. (1996). Inventory of instruments of critical thinking. 

Informal Logic, 18(2). https://doi.org/10.22329/il.v18i2.2383 
 
Genesereth, M. & Chaudhri, V.  (n.d.). Logic in secondary education. Stanford University. 

Retrieved November 3, 2021, from http://intrologic.stanford.edu/miscellaneous/intro.html 
Glantz, S. A. (2012). Primer of biostatistics: Seventh edition. The McGraw-Hill Companies, Inc.  
 
History of Logic | Ancient, medieval, modern, & contemporary logic. (n.d.). Encyclopedia 

Britannica. Retrieved November 29, 2021, from 
https://www.britannica.com/topic/history-of-logic 

 
Ikuenobe, P. (2001). Teaching and assessing critical thinking abilities as outcomes in an informal 

logic course. Teaching in Higher Education, 6(1), 19–32. https://doi-
org.tcsedsystem.idm.oclc.org/10.1080/13562510020029572 

 
Kobylarek, A. (2020). Power as knowledge. The reverse logic of the post-scientific world. 

Journal of Education, Culture & Society, 11(2), 5–14. https://doi-
org.tcsedsystem.idm.oclc.org/10.15503/jecs2020.2.5.14 

 

https://doi.org/10.14689/ejer.2015.59.11
https://cimt.org.uk/journal/bakom.pdf
https://futurefocusedlearning.net/blog/critical-thinking/5-of-the-best-practices-for-improving-critical-thinking-skills
https://futurefocusedlearning.net/blog/critical-thinking/5-of-the-best-practices-for-improving-critical-thinking-skills
https://doi.org/10.22329/il.v18i2.2383
http://intrologic.stanford.edu/miscellaneous/intro.html
https://www.britannica.com/topic/history-of-logic
https://doi-org.tcsedsystem.idm.oclc.org/10.1080/13562510020029572
https://doi-org.tcsedsystem.idm.oclc.org/10.1080/13562510020029572
https://doi-org.tcsedsystem.idm.oclc.org/10.15503/jecs2020.2.5.14
https://doi-org.tcsedsystem.idm.oclc.org/10.15503/jecs2020.2.5.14


2 

Morozova, E. (2014). Development of the system of pedagogical monitoring of students’ logical 
thinking and logical reflection. Concept / Koncept, 10, 1–7. 

 
Perdue, S. M. (2014, September 19). The big three of greek philosophy: Socrates, plato, and 

aristotle. | English Blog. Penn State. Retrieved November 29, 2021, from 
https://sites.psu.edu/rclperdue/2014/09/19/the-big-three-of-greek-philosophy-socrates-
plato-and-aristotle/ 

 
Piaget, J. (1999). The child’s conception of physical causality (1st ed.). Routledge. 
 
Publication manual of the American Psychological Association (7th ed.). (2020). In American 

Psychological Association eBooks. https://doi.org/10.1037/0000165-000 
 
Robinson, S. R. (2011). Teaching logic and teaching critical thinking: revisiting McPeck. Higher 

Education Research and Development, 30(3), 275–287. 
https://doi.org/10.1080/07294360.2010.500656 

 
Rochat, P. (2024). The evolution of developmental theories since piaget: A metaview. 

Perspectives on Psychological Science, 19(6), 921–930. 
https://doi.org/10.1177/17456916231186611 

 
Sherman, R. (2021). Logical fallacies. [Unpublished manuscript]. Department of Applied 

Psychophysiology, Saybrook University. 
 
The sophists; Stanford encyclopedia of philosophy. (2020, August 18). Stanford Encyclopedia of 

Philosophy. Retrieved November 29, 2021, from 
https://plato.stanford.edu/entries/sophists/ 

 
Tobin, K.G., & Capie, W. (1981). The development and validation of a group test of logical 

thinking. Educational and psychological measurement, 41(2), 413–423.  
 
Well, T., PhD. (2023, July 5). The decline of critical thinking skills. Psychology Today. 

https://www.psychologytoday.com/us/blog/the-clarity/202306/the-decline-of-critical-
thinking-skills 

 
UNC-Chapel Hill Writing Center. (2021, April 14). Fallacies. The Writing Center at University 

of North Carolina at Chapel Hill. Retrieved November 29, 2021, from 
https://writingcenter.unc.edu/tips-and-tools/fallacies/  

 

 

 

 

https://sites.psu.edu/rclperdue/2014/09/19/the-big-three-of-greek-philosophy-socrates-plato-and-aristotle/
https://sites.psu.edu/rclperdue/2014/09/19/the-big-three-of-greek-philosophy-socrates-plato-and-aristotle/
https://doi.org/10.1037/0000165-000
https://doi.org/10.1080/07294360.2010.500656
https://doi.org/10.1177/17456916231186611
https://plato.stanford.edu/entries/sophists/
https://www.psychologytoday.com/us/blog/the-clarity/202306/the-decline-of-critical-thinking-skills
https://www.psychologytoday.com/us/blog/the-clarity/202306/the-decline-of-critical-thinking-skills
https://writingcenter.unc.edu/tips-and-tools/fallacies/

