































   Advancements in Agricultural Development 
  Volume 6, Issue 4, 2025 
  agdevresearch.org 

 

1. K. S. U. Jayaratne, Professor and State Leader for Extension Evaluation, Department of Agricultural and Human Sciences, 
North Carolina State University, 240 Ricks Hall, Campus Box 7607, Raleigh, NC 27695, USA, jay_jayaratne@ncsu.edu,        

 https://orcid.org/0000--0000-0000-0000 
2. Anil Kumar Chaudhary, Associate Professor, Department of Agricultural Economics, Sociology, and Education, Pennsylvania 

State University, University Park, PA, 16802, USA. auk259@psu.edu,  https://orcid.org/0000-0001-5809-8854  
3. John M. Diaz, Associate Professor and Extension Specialist, Department of Agricultural Education and Communication, 

University of Florida, Plant City, FL 33563, USA, john.diaz@ufl.edu,  http://orcid.org/0000-0002-2787-8759  
 

64 

 

Knowledge Testing Options in Pre-Test Post-Test Evaluation 
Design: Implications for Extension Program Evaluation 

 
K. S. U. Jayaratne1, A. Kumar Chaudhary2, J. M. Diaz3 

 
 

Article History 
Received: August 15, 2025 
Accepted: October 28, 2025 
Published: November 17, 2025 
 
 
Keywords 
knowledge testing; objective 
knowledge test; subjective knowledge 
test; retrospective pre-test; extension 
evaluation; SDG 3: Good Health and 
Well-being 
  

Abstract 
Knowledge gained is an important outcome indicator used in the 
evaluation of extension education programs.  Three evaluation designs 
commonly used to document extension program participants’ knowledge 
change are: (a) objective knowledge pre-test, post-test design, (b) 
subjective knowledge pre-test, post-test design, and (c) subjective 
knowledge retrospective pre-test, post-test design. This study was 
designed to examine the relationship between measuring variables of 
objective knowledge pre-test, post-test, and subjective knowledge pre-
test, post-test designs, and to examine the validity of subjective 
knowledge pre-test and post-test designs in assessing the knowledge 
gained by the Cook Smart Eat Smart extension participants. The 
researchers developed the survey instrument to document objective 
knowledge and subjective knowledge. The survey was administered 
before and after completing the training. The study received 71 
responses. Paired sample t-test and correlation analysis were used to 
achieve research objectives. The findings indicate that all three designs 
are effective in documenting the changes in participants’ knowledge. 
Findings also verified the accuracy and validity of the subjective 
knowledge retrospective pre-test and post-test design.  The evaluation of 
extension programs is strengthened by evidence demonstrating the 
validity of both objective and subjective measures for assessing 
knowledge gain, a key outcome indicator in extension education. 

 

mailto:jay_jayaratne@ncsu.edu
https://orcid.org/0000-0001-5591-3905
https://orcid.org/0000-0001-5591-3905
mailto:auk259@psu.edu
https://orcid.org/0000-0001-5809-8854
mailto:john.diaz@ufl.edu
http://orcid.org/0000-0002-2787-8759


Jayaratne et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i4.659   65 
 

Introduction and Problem Statement 
 
Documenting knowledge change is central to evaluating extension programs because learning 
often influences behavior and long-term outcomes (Bennett, 1975). Extension professionals, 
therefore, must consider how to best measure knowledge improvement when assessing 
program effectiveness (Diaz et al., 2021). 
 
Objective pre-test and post-test designs are often considered the most rigorous approach. 
Objective knowledge reflects what participants actually know, measured through factual tests 
(Brucks, 1985). Capturing changes between two points provides strong evidence of learning 
(Ary et al., 2006; O’Leary & Israel, 2019). However, these instruments require considerable time 
to develop and administer. In shorter programs, testing may also reduce instructional time and 
create participant fatigue, especially when surveys are lengthy. 
 
To address these challenges, many extension professionals employ subjective pre-test and 
post-test measures, which assess what participants believe they know (Brucks, 1985). Findings 
regarding how subjective and objective knowledge align are inconsistent. Several studies report 
moderate to strong positive correlations (Brucks, 1985; Raju et al., 1995), while others report 
weak or no correlation (Carlson et al., 2009; Ellen, 1994; Farrell et al., 2010). This inconsistency 
raises concerns about validity (Joshi et al., 2024). 
 
The lack of consensus creates a problem for extension evaluation: subjective measures are 
practical but may not accurately reflect actual knowledge change. This uncertainty limits 
professionals’ ability to select designs that balance feasibility with validity. In response to this 
gap, the current investigation evaluates the effectiveness of subjective pre-test and post-test 
designs relative to objective testing for measuring knowledge gains in the Cook Smart, Eat 
Smart extension program. 
 

Conceptual Framework 
 
With knowledge as a construct defined as “the information stored within memory” (Engel et al., 
1993, p. 281), the subjective knowledge assessments are designed to record training 
participants’ perceived knowledge about the subject (Spreng & Olshavsky, 1990). The subjective 
knowledge testing instruments are designed with some statements related to the testing of 
knowledge and recording participants’ level of responses on a Likert scale. This design requires 
developing a knowledge baseline through the administration of a pre-test at the beginning of 
the training and a post-test at the end of the training to assess knowledge gains, which creates 
an additional onus of time on participants to complete the evaluation. Therefore, this design 
may be time-intensive and not suitable for the assessment of short training programs. Another 
limitation of this subjective pre-test and post-test design is its possible measurement errors due 
to response-shift bias of participants (Chasteen & Chattergoon, 2019; Moore & Tananis, 2009; 
Thomas et al., 2018). A response-shift bias occurs when individuals inaccurately rate their own 
knowledge on a pre-test because they lack sufficient understanding of the subject to provide an 

https://doi.org/10.37433/aad.v6i4.659


Jayaratne et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i4.659   66 
 

informed assessment (Nielsen, 2011). This is a serious issue when extension agents deliver 
programs on unfamiliar topics because participants often lack the necessary context to 
accurately gauge their prior knowledge, which can lead them to rate their perceived knowledge 
as high at the outset before being exposed to the content. After completing the training, they 
realized that they knew only a little about the subject and rated relatively low on the post-test. 
This shifting of response bias may fail to capture changes in knowledge that occur between pre-
test and post-test assessments. It is a problem with the design due to the shifting response bias 
of participants (Chasteen & Chattergoon, 2019; Nielsen, 2011).  
 
Extension professionals use the subjective knowledge retrospective pre-test and post-test 
design as an alternative to overcome the above-discussed response-shift bias of subjective 
knowledge testing pre-test and post-test design. Retrospective pre-test post-test design, also 
known as the then-test design, provides the same frame of reference to measuring variables 
and controls the response shifting bias (Drennan & Hyde, 2008; Nielsen, 2011; Taminiau-Bloem 
et al., 2015). Providing the same frame of reference is a requirement to consider that pre-test 
and post-test comparisons are valid (Howard, 1980). Additionally, retrospective pre-test and 
post-test evaluation design is more sensitive to changes in respondents than traditional pre-test 
and post-test evaluations (Skeff et al., 1992). The retrospective subjective pre-test/post-test 
design is becoming an increasingly popular option for evaluating extension programs, 
particularly because it only requires administration once at the end of the training. This 
approach is considered more accurate for capturing self-reported knowledge gains, as 
participants can reflect on both their baseline knowledge and what they learned after 
completing the program. Researchers have recommended this method for assessing short-term 
training programs, as it avoids the need for pre-program administration and preserves more 
time for instruction (Nielsen, 2011). Retrospective pre-test documents self-reported data, 
leading to generate an estimated evaluation report (Pratt et al., 2000), which is different from 
an actual (objective) knowledge test report. 
 
The available literature (Chasteen & Chattergoon, 2019; Taminiau-Bloem et al., 2015; Thomas 
et al., 2018) is inconsistent in the validity of these subjective knowledge pre-test and post-test 
designs compared to the objective knowledge pre-test and post-test design. This study aims to 
test the validity of the subjective knowledge pre-test and post-test design, and the subjective 
knowledge retrospective pre-test and post-test design compared to the objective knowledge 
pre-test and post-test design. Figure 1 provides the conceptual frame of comparing three 
designs of knowledge testing, i.e., 1) objective knowledge pre-test post-test design, 2) 
subjective knowledge pre-test post-test design, and 3) subjective knowledge retrospective pre-
test post-test design. 
 
 
 
 
 
 

 

https://doi.org/10.37433/aad.v6i4.659


Jayaratne et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i4.659   67 
 

 
Figure 1 

Comparison of the Effectiveness of Three Designs of Evaluations for Assessing Participants’ 

Knowledge Gained  

 

 

 

 

 

 

 

 

Purpose 
 
The present research aimed to compare the validity of three knowledge testing evaluation 
designs in the context of the Cook Smart, Eat Smart extension program. By examining the 
strengths and limitations of each design, this study provides insights that can guide 
practitioners in selecting evaluation approaches that balance rigor with feasibility in program 
settings. More precisely, this research sought to address these objectives: 
1. To compare the validity of the objective knowledge pre-test and post-test design, the 

subjective knowledge pre-test and post-test design, and the subjective retrospective pre-
test and post-test design in evaluating knowledge gains as a result of the Cook Smart Eat 
Smart extension program. 

2. To determine the correlation between objective knowledge measurements and subjective 
knowledge measurements at pre-tests, post-tests, and retrospective pre-tests. 

3. Determine the correlation between objective knowledge gained, subjective knowledge 
gained, and the retrospectively assessed subjective knowledge gained of Cook Smart Eat 
Smart participants. 

Objective Pretest Objective Posttest 

3.  Subjective (Retrospective 
pretest) Testing of Knowledge 
Gained 

2.   Subjective Testing of Knowledge 
Gained 

1. Objective Testing of Knowledge 
Gained 

Subjective 
(Retrospective) 

Pretest 

Subjective 
Posttest 

Subjective 
Posttest 

Subjective 
Pretest 

Controlled Response Shift Bias 

Correlation 

Correlation 

 

Correlation 

Correlation 

 

Response Shifting Bias  

https://doi.org/10.37433/aad.v6i4.659


Jayaratne et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i4.659   68 
 

4. To assess the relative accuracy of the subjective (perceived) pre-test and post-test 
knowledge assessment design and the retrospective pre-test and post-test design in 
evaluating an extension program. 

 
Methods 

 
This evaluation study was conducted using the Cook Smart, Eat Smart extension program. Three 
Family and Consumer Sciences (FCS) County Extension Agents were selected based on 
recommendations from the state FCS program leader. Each of the selected agents had prior 
experience delivering the Cook Smart, Eat Smart curriculum. This multi-session program is 
designed to help participants build practical cooking knowledge and skills that can be applied at 
home. The first author reached out to the agents in North Carolina State to explain the purpose 
of the study, outline the key data collection points, and emphasize the importance of collecting 
complete and accurate data. 
 
Researchers developed the required evaluation instruments by reviewing the Cook Smart Eat 
Smart curriculum and relevant literature to collect research data. The pre- and post-survey 
instruments were pilot tested with a similar group of training participants to establish face 
validity. A panel of extension and evaluation experts reviewed the instrument and established 
content validity. The data collection instruments included the pre-test instrument and the post-
test instrument. The pre-test instrument included two components: a 25-item objective 
knowledge test using true/false/don’t know questions based on the curriculum, and a 15-item 
subjective knowledge assessment using a 5-point Likert scale. The Likert scale ranged from 1 = 
very low to 5 = very high knowledge. The post-test survey consisted of three sections: an 
objective knowledge testing post-test, a subjective knowledge testing post-test, and a 
subjective knowledge testing retrospective pre-test and post-test. These are the same 
questions used in the subjective knowledge testing pre-test and the post-test.  The score on the 
objective knowledge testing 25-factual-question tests can range from 0 = very low to 25 = very 
high. The subjective knowledge testing score recording on the 15-item scale can range from 15 
= very low to 75 = very high. Cronbach’s reliability Alpha of the subjective knowledge testing 
scale was .91.  
 
The three selected Extension Agents were responsible for scheduling, advertising, and 
recruiting participants for the program. One agent conducted two training programs, enrolling 
16 participants in one and 18 in the other, for a total of 34 individuals. The other two agents 
recruited 16 and 21 participants, respectively, for their programs. Each agent delivered the 
Cook Smart, Eat Smart curriculum across multiple sessions. The consent form and the pre-test 
were given at the beginning of the training program, and the post-test was administered at the 
end of the training. The study received 71 responses. 
 
Data were analyzed using IBM SPSS 26®. We used paired sample t-test analysis and the Pearson 
product-moment correlation coefficient to accomplish objectives. We used Meghanathan’s 
(2016) classification of correlation coefficients to describe our findings (see Table 1). 

https://doi.org/10.37433/aad.v6i4.659


Jayaratne et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i4.659   69 
 

Table 1  
 
Range of Correlation Coefficient Values and the Corresponding Descriptors 
Range of Correlation Coefficient Values Description 
+/- .80 - 1.00 Very strong 
.60 - .79 Strong 
.40 - .59 Moderate 
.20 - .39 Weak 
 - .19 Very weak 

 
Findings 

 
Findings are structured according to four primary objectives. 
 
Objective 1: The Comparison of the Validity of Three Designs in Evaluating the Change in 
Knowledge of Cook Smart Eat Smart Participants  
Paired-samples t tests were conducted to analyze data related to the first research objective. 
The paired-samples t-test findings in Table 2 indicate that the objective knowledge pre-test and 
post-test design, the subjective knowledge pre-test and post-test design, and the subjective 
knowledge test with retrospective pre-test and subjective knowledge post-test design are 
effective evaluation designs for measuring knowledge change (i.e., gain) among the Cook 
Smart, Eat Smart program participants. All three evaluation designs recorded a significant 
knowledge gain by participants from pre-test to post-test. Participants’ knowledge gain was 
highly visible in the subjective knowledge retrospective pre-test and post-test design compared 
to the other two designs (see Table 2). The mean value of the subjective knowledge pre-test 
(45.1) was higher than the mean value of the subjective knowledge retrospective pre-test 
(43.4). This may be due to response shift bias (Chasteen & Chattergoon, 2019; Moore & 
Tananis, 2009; Thomas et al., 2018) in the subjective knowledge pre-test. 
 
Table 2  
 
The Comparison of Objective and Subjective Assessment of Knowledge Before and After 
Completing the Training 
 

N 
M 

t p Type of knowledge testing design Pre-test Post-test 
Objective knowledge pre-test and post-test 69 16.1 18.9 6.03 .001** 
Subjective knowledge pre-test and post-test 58 45.1 59.0 7.86 .001** 
Subjective knowledge test with retrospective 

pre-test and subjective post-test 
52 43.4 59.0 11.34 .001** 

Note. P**<.001 
 
 
 

https://doi.org/10.37433/aad.v6i4.659


Jayaratne et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i4.659   70 
 

Objective 2: The Correlation Between Knowledge Test Scores of Three Designs 
A moderate positive correlation was found between the objective pre-test and post-test scores, 
r = .43, p < .05 (see Table 3). In contrast, the correlation between subjective and objective pre-
test scores was not significant, suggesting that the subjective pre-test did not reliably reflect 
participants’ actual baseline knowledge. A weak but significant positive correlation was 
observed between subjective and objective post-test scores, r(n-2) = .32, p < .05. 
 
Table 3  
 
Pearson’s Correlation Between Objective Assessment and Subjective Assessment of Knowledge 
Testing Scores 
Variable n M SD 1 2 3 4 5 
Objective knowledge pre-test 

score 
71 15.9 4.13 -     

Objective knowledge post-test 
score 

69 18.9 2.91 .43** -    

Subjective knowledge pre-test 
score 

62 45.2 11.0 .12 .02 -   

Subjective knowledge post-test 
score 

63 59.1 8.55 .43** .32* .119 -  

Subjective knowledge 
retrospective pre-test score 

58 43.7 9.20 .40** .26* .45** .41** - 

Note. P*<.05, P**<.001 
 
A significant moderate positive correlation was found between the retrospective subjective 
pre-test score and the objective pre-test score, r(n – 2) = .40, p < .001. The retrospective 
subjective pre-test score was also weakly but positively correlated with the objective post-test 
score, r(n – 2) = .26, p < .05. In addition, moderate positive correlations emerged between the 
retrospective subjective pre-test score and the subjective pre-test score, r(n – 2) = .45, p < .001, 
as well as between the retrospective subjective pre-test score and the subjective post-test 
score, r(n – 2) = .41, p < .001. 
 
Objective 3: The Correlation Between Participants’ Knowledge Gains in Three Evaluation 
Designs 
Knowledge gain was calculated by subtracting pre-test scores from post-test scores for each 
evaluation design. This produced gain scores for the objective pre-test–post-test, the subjective 
pre-test–post-test, and the retrospective subjective pre-test–post-test designs. Correlation 
analyses were then conducted to examine the relationships among these three gain measures 
(see Table 4). 
 
 
 
 
 

https://doi.org/10.37433/aad.v6i4.659


Jayaratne et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i4.659   71 
 

Table 4  
 
Pearson Correlations of Objective and Subjective Knowledge Measures 
Variable n M SD 1 2 3 
Objective knowledge change between pre-

test and post-test 
69 2.8 3.87 -   

Subjective knowledge change between 
post-test and pre-test 

58 13.8 13.46 .04 -  

Subjective knowledge change between the 
retrospective pre-test and the post-test 

52 15.5 9.88 .04 .59** - 

Note. P**<.001 
 
Table 4 shows that objective knowledge change was not significantly correlated with subjective 
knowledge change, including knowledge gains measured through the retrospective pre-test–
post-test design. However, moderate positive correlations were observed among subjective 
knowledge gains across the traditional pre-test–post-test and retrospective pre-test–post-test 
measures. 
 

Conclusions, Discussion, and Recommendations  
 

This study examined the validity of three evaluation designs—objective pre-test/post-test, 
subjective pre-test/post-test, and subjective retrospective pre-test/post-test—in measuring 
change in knowledge among participants in the Cook Smart, Eat Smart Extension program. The 
findings confirm that all three designs were effective in documenting statistically significant 
knowledge gains, but their accuracy, practicality, and alignment with one another varied. 
 
The objective knowledge pre-test/post-test design, often considered the most rigorous method 
for evaluating learning outcomes, directly measured factual knowledge using a set of 25 
true/false questions with a “don’t know” option to reduce guessing error. This approach clearly 
demonstrated significant improvement in knowledge. However, its practical limitations, 
including the time required to develop the instrument, administer pre- and post-tests, and 
engage participants, make it less feasible for short programs or time-constrained settings. 
Participants’ reluctance to complete lengthy objective assessments is another drawback that 
may impact data quality and response rates. 
 
The subjective knowledge pre-test/post-test design also showed significant increases in 
perceived knowledge. However, its accuracy is questionable. The absence of a significant 
correlation between subjective and objective pre-test scores raises concerns about participants’ 
ability to accurately assess their baseline knowledge before exposure to the content. This 
misalignment is likely due to response shift bias, where participants overestimate their 
knowledge at the outset because they are unaware of what they do not yet know (Chasteen & 
Chattergoon, 2019; Moore & Tananis, 2009; Thomas et al., 2018). Although there was a weak 
but statistically significant correlation between subjective and objective post-test scores, the 

https://doi.org/10.37433/aad.v6i4.659


Jayaratne et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i4.659   72 
 

inconsistency between pre- and post-test measures undermines the credibility of this design 
when used alone. 
 
In contrast, the subjective retrospective pre-test/post-test design not only recorded significant 
knowledge gains but also demonstrated more consistent alignment with objective knowledge 
measures. A moderate positive correlation was observed between the retrospective pre-test 
and objective pre-test scores (.40), and a weak correlation between the objective post-test and 
subjective post-test scores (.32), indicating participants' pre and post-program reflections were 
reasonably accurate compared to their actual prior knowledge and perceptions. Similarly, the 
moderate correlation between the retrospective pre-test and the subjective post-test (.41) 
further supports the internal consistency of this design. Although the correlation between 
subjective post-test and objective post-test scores was weak (.32), it still suggests some 
alignment between perceived and actual knowledge after the program. 
 
These results provide strong support for the retrospective design as a practical and valid 
alternative, especially in settings where administering traditional pre-tests is not practical. The 
design effectively avoids the response shift bias common in standard subjective pre-test/post-
test option (Drennan & Hyde, 2008; Nielsen, 2011; Taminiau-Bloem et al., 2015) and is easier to 
develop and administer. Its weak to moderate alignment with objective measures strengthens 
its credibility as a practical design in extension program evaluation. 
 
Importantly, the analysis revealed no significant correlation between objective knowledge 
change and either form of subjective knowledge change. This disconnect reinforces concerns 
about the credibility of self-reported data in capturing actual learning gains. Participants may 
misjudge the extent of their improvement, either underestimating or overestimating their 
learning. Despite this, the moderate correlation between changes recorded by the subjective 
pre-test/post-test and retrospective pre-test/post-test designs suggests consistency in 
participants' perceptions of knowledge gain across both methods. 
 
Taken together, these findings suggest that while objective evaluations remain the gold 
standard in terms of accuracy, the subjective knowledge retrospective pre-test and post-test 
design offers a highly practical and sufficiently credible alternative. When time, resources, or 
participant burden limit the use of objective methods, the retrospective design can provide 
meaningful insights into program effectiveness. It is especially well-suited for short-duration or 
introductory programs where participants lack the baseline knowledge needed to accurately 
assess themselves at the outset. 
 
Recommendations and Implications  
When conditions allow, the objective pre-test/post-test design should be used for its accuracy 
in capturing factual knowledge gain. However, in programs with time constraints or limited 
participant engagement, the retrospective pre-test/post-test design offers a more practical and 
credible alternative. The traditional subjective pre-test/post-test design may be used 
cautiously, keeping in mind the potential for response shift bias (Chasteen & Chattergoon, 

https://doi.org/10.37433/aad.v6i4.659


Jayaratne et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i4.659   73 
 

2019; Moore & Tananis, 2009; Thomas et al., 2018) and the challenges participants face in 
evaluating their knowledge before exposure to new content. 
 
The most significant implication of this study is that it affirms the validity and practicality of the 
subjective knowledge retrospective pre-test and post-test design as an effective evaluation tool 
for extension programs. This design strikes a balance between accuracy and feasibility and can 
improve the consistency and credibility of evaluation results across various program formats 
and audiences. 
 
Limitations and Future Research  
The research was conducted with a relatively small sample of participants from a single 
extension program in one state, which limits the generalizability of the findings to other 
programming contexts. Future research should replicate this study across multiple program 
areas, such as agriculture and 4-H, and in different states or regions to assess the broader 
applicability of these evaluation designs. Additional work is also needed to explore how factors 
such as audience characteristics, program duration, and content complexity influence the 
alignment between perceived and actual knowledge gains. 
 

Acknowledgments 
 

Funding Information: The second author secured funding for this research from Pennsylvania 
State University’s Penn State Extension through its Multistate and Integrated Program Grants. 
 
Conflict of interest: There are no conflicts of interests.  
 
Previous Dissemination: Any portion of this article or research was not previously included in a 
thesis, dissertation, conference paper, and/or abstract.  
 
Artificial Intelligence: Artificial intelligence tools were not used in this study.  
 
Author Contribution Statement: K. S. U. Jayaratne – Contributed to conceptualization of the 
study, methodology, instrumentation, investigation, analysis of data, writing original draft, and 
review editing; A. K. Chaudhary – Contributed to funding acquisition, conceptualization of the 
study, methodology, writing original draft, and review editing; J. M. Diaz – Contributed to 
conceptualization of the study, methodology, writing original draft, and review editing. 
 

References  
Ary, D., Jacobs, L. C., Razavieh, A., & Sorensen, C. (2006). Introduction to research in education 

(7th ed.). Thomson Wadsworth.  

Bennett, C. (1975). Up the hierarchy. The Journal of Extension, 13(2), Article 2. 
https://open.clemson.edu/joe/vol13/iss2/2  

https://doi.org/10.37433/aad.v6i4.659
https://open.clemson.edu/joe/vol13/iss2/2


Jayaratne et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i4.659   74 
 

Brucks, M. (1985). The effects of product class knowledge on information search behavior. 
Journal of Consumer Research, 12(1), 1-16. https://doi.org/10.1086/209031  

Carlson, J. P., Vincent, L. H., Hardesty, D. M., Bearden, W. O., & Deighton, J. (2009). Objective 
and subjective knowledge relationships: A quantitative analysis of consumer research 
findings. Journal of Consumer Research, 35(5), 864–876. 
https://doi.org/10.1086/593688  

Chasteen, C., & Chattergoon, R. (2019). A comparison study of pre/post-test and retrospective 
pre-test for measuring faculty attitude change. 2019 PERC Proceedings, 93-98. 
https://doi.org/10.1119/perc.2019.pr.Chasteen  

Diaz, J., Silvert, C., Gusto, C., Jayaratne, K., & Narine, L. (2021). Towards intercultural 
competence: Using consensus to identify essential personality traits for an inclusive 
extension education workforce. Advancements in Agricultural Development, 2(3), 83–95. 
https://doi.org/10.37433/aad.v2i3.135  

Drennan, J., & Hyde, A. (2008). Controlling response shift bias: The use of the retrospective pre-
test design in the evaluation of a master's programme. Assessment & Evaluation in 
Higher Education, 33(6), 699-709. https://doi.org/10.1080/02602930701773026  

Ellen, P. S. (1994). Do we know what we need to know? Objective and subjective knowledge 
effects on pro-ecological behaviors. Journal of Business Research, 30(1), 43–52. 
https://doi.org/10.1016/0148-2963(94)90067-1  

Engel, J. F., Blackwell, R. D., & Miniard, P. W. (1993). Consumer behavior (7th ed.). The Dryden 
Press. 

Farrell, T., Kohn, G. P., Owen, S. M., Meyers, M. O., Stewart, R. A., & Meyer, A. A. (2010). Low 
correlation between subjective and objective measures of knowledge on surgery 
clerkships. Journal of the American College of Surgeons 210(5), 680-683. 
https://doi.org/10.1016/j.jamcollsurg.2009.12.020  

Howard, G. S. (1980). Response-shift bias: A problem in evaluating interventions with pre/post 
self-reports. Evaluation Review, 4(1), 93–106. 
https://doi.org/10.1177/0193841X8000400105  

Joshi, A., Diaz, J., Kumar Chaudhary, A., Jayaratne, K. S. U., & Galindo, S. (2024). Enhancing 
effectiveness of Extension program evaluations by validating the trustworthiness of self-
reported measures of Extension program outcomes. Advancements in Agricultural 
Development, 5(4), 59–71. https://doi.org/10.37433/aad.v5i4.519  

Meghanathan, N. (2016). Assortativity analysis of real-world network graphs based on centrality 
metrics. Computer and Information Science, 9(3),7-25. 
https://doi.org/10.5539/cis.v9n3p7  

https://doi.org/10.37433/aad.v6i4.659
https://doi.org/10.1086/209031
https://doi.org/10.1086/593688
https://doi.org/10.1119/perc.2019.pr.Chasteen
https://doi.org/10.37433/aad.v2i3.135
https://doi.org/10.1080/02602930701773026
https://doi.org/10.1016/0148-2963(94)90067-1
https://doi.org/10.1016/j.jamcollsurg.2009.12.020
https://doi.org/10.1016/j.jamcollsurg.2009.12.020
https://doi.org/10.1177/0193841X8000400105
https://doi.org/10.37433/aad.v5i4.519
https://doi.org/10.5539/cis.v9n3p7


Jayaratne et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v6i4.659   75 
 

Moore, D., & Tananis, C. A. (2009). Measuring change in a short-term educational program 
using a retrospective pretest design. American Journal of Evaluation, 30(2), 189–202. 
https://doi.org/10.1177/1098214009334506  

Nielsen, R. B. (2011). A Retrospective pretest-posttest evaluation of a one-time personal 
finance training. Journal of Extension 49(1), Article 4. 
https://doi.org/10.34068/joe.49.01.04  

O’Leary, J. L., & Israel, G. D. (2019). Capturing change: Comparing pretest-posttest and 
retrospectivee methods. IFAS Extension, University of Florida. 
https://doi.org/10.32473/edis-wc135-2013  

Pratt, C. C., McGuigan, W. M., & Katzev, A. R. (2000). Measuring program outcomes: Using 
retrospective pretest methodology. American Journal of Evaluation, 21(3), 341–349. 
https://doi.org/10.1177/109821400002100305  

Raju, P. S., Lonial, S. C., & Mangold, W. G. (1995). Differential effects of subjective knowledge, 
objective knowledge, and usage experience on decision making: An exploratory 
investigation. Journal of Consumer Psychology, 4(2), 153-180. 
https://doi.org/10.1207/s15327663jcp0402_04  

Skeff, K. M., Stratos, G. A., & Bergen, M. R. (1992). Evaluation of a medical faculty development 
program: A comparison of traditional pre/post and retrospective pre/post self-
assessment ratings. Evaluation and the Health Care Professions, 15(3), 350-366. 
https://doi.org/10.1177/016327879201500307  

Spreng, R. A., & Olshavsky, R. W. (1990). Exploring the headwaters of the prior knowledge—
search relationship. In A. Parasuraman & W. O. Bearden (Eds.), Enhancing knowledge 
development in marketing (pp. 220–224). American Marketing Association. 

Taminiau-Bloem, E. F., Schwartz, C. E., van Zuuren, F. J., Koeneman, M. A., Visser, M. R. M., 
Tishelman, C., Koning, C. C. E., & Sprangers, M. A. G. (2015). Using a retrospective 
pretest instead of a conventional pretest is replacing biases: A qualitative study of 
cognitive processes underlying responses to thentest items. Quality of Life Research. 
25(6), 1327-1337. https://doi.org/10.1007/s11136-015-1175-4  

Thomas, E. V., Wells, R., Baumann, S. D., Graybill, E., Roach, A., Truscott, S. D., Crenshaw, M., & 
Crimmins, D. (2018). Comparing traditional versus retrospective pre-/post-assessment in 
an interdisciplinary leadership training program. Maternal and Child Health Journal 23, 
191–200. https://doi.org/10.1007/s10995-018-2615-x  

 

© 2025 by authors. This article is an open access article distributed under the terms and conditions of 
the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/). 
 

https://doi.org/10.37433/aad.v6i4.659
https://doi.org/10.1177/1098214009334506
https://doi.org/10.34068/joe.49.01.04
https://doi.org/10.32473/edis-wc135-2013
https://doi.org/10.1177/109821400002100305
https://doi.org/10.1207/s15327663jcp0402_04
https://doi.org/10.1177/016327879201500307
https://doi.org/10.1007/s11136-015-1175-4
https://doi.org/10.1007/s10995-018-2615-x

