Advancements in Agricultural Development Volume 5, Issue 4, 2024 agdevresearch.org 1. Arati Joshi, Graduate Student, Department of Agricultural Education and Communication, University of Florida, 408 Rolfs Hall, 341 Buckman Dr, Gainesville, FL, 32611, aratijoshi@ufl.edu, https://orcid.org/0000-0002-4946-9366 2. John Diaz, Associate Professor & Extension Specialist, Department of Agricultural Education and Communication, University of Florida, 1200 N Park Road, Plant City, FL 33563, john.diaz@ufl.edu, https://orcid.org/0000-0002-2787-8759 3. Anil Kumar Chaudhary, Associate Professor of Agricultural and Extension Education, Pennsylvania State University, 386 Shortlidge Rd, 209C Ferguson Building, University Park, PA 16802, auk259@psu.edu, https://orcid.org/0000-0001-5809-8854 4. K. S. U. Jayaratne, Professor and State Leader for Extension Evaluation, 240 Ricks Hall, Campus Box 7607, Department of Agricultural and Human Sciences, North Carolina State University, Raleigh, NC 27695, ksjayara@ncsu.edu, https://orcid.org/0000-0001-5591-3905 5. Sebastian Galindo, Associate Professor of Program Evaluation, Department of Agricultural Education and Communication, University of Florida, 113C Bryant Hall, Gainesville, FL 32611, sgalindo@ufl.edu, https://orcid.org/0000-0002-4061-773X 59 Enhancing Effectiveness of Extension Program Evaluations by Validating the Trustworthiness of Self-Reported Measures of Extension Program Outcomes A. Joshi1, J. Diaz2, A. Kumar Chaudhary3, K. Jayaratne4, S. Galindo5 Article History Received: July 10, 2024 Accepted: November 12, 2024 Published: November 15, 2024 Keywords subjective knowledge; objective knowledge; behavior change; impact assessment; SDG 15: Life on Land Abstract Assessment of program outcomes in extension often relies on subjective measures, such as perceived or self-reported knowledge, which are criticized for potential bias and inaccuracy. Conversely, objective knowledge, i.e., how much an individual actually knows, is considered more accurate. Studies show varying associations between subjective and objective knowledge, ranging from no correlation to high correlation, and their influence on behavior change also varies. In this study, we aim to quantify the relationship between subjective knowledge, objective knowledge, and behavior change. Data were collected from Master Gardener Volunteer training attendees. We used Pearson correlation and hierarchical linear regressions to explore the relationship between subjective and objective knowledge and their influence on behavior, i.e., engagement in gardening practices. Our findings show that subjective and objective knowledge post-training were moderately correlated, indicating that participants' self- assessments were not entirely accurate before training. Interestingly, only subjective knowledge before training predicted engagement in gardening practices after training, highlighting the significant role of perceived understanding in behavior change. Based on the findings, we suggest that extension programs should focus on addressing participants' existing beliefs to foster enduring behavior change. By designing programs that consider these pre- existing perceptions, extension can more effectively translate knowledge into practical, lasting behaviors. mailto:aratijoshi@ufl.edu https://orcid.org/0000-0002-4946-9366 mailto:john.diaz@ufl.edu https://orcid.org/0000-0002-2787-8759 mailto:auk259@psu.edu https://orcid.org/0000-0001-5809-8854 mailto:ksjayara@ncsu.edu https://orcid.org/0000-0001-5591-3905 mailto:sgalindo@ufl.edu https://orcid.org/0000-0002-4061-773X Joshi et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i4.519 60 Introduction and Problem Statement Evaluating the impact of extension programs and measuring knowledge and behavior change is a complex task (Larese-Casanova, 2017). There is ongoing debate regarding the use of subjective versus objective measures to evaluate these changes. Subjective measures, where participants self-report their knowledge and awareness before and after program participation, are commonly used due to their practicality, ease of administration, and broad applicability (Aqueveque, 2018; Gonyea, 2005). However, research suggests that relying solely on subjective assessments may lead to cognitive bias, where participants may either underestimate or overestimate their actual knowledge (Han, 2019; House et al., 2004; Kruger & Dunning, 1999; O’Leary & Israel, 2019). Other researchers argue for the inclusion of objective measures, such as quizzes, tests, or skills assessments, alongside subjective assessments, as they more accurately gauge knowledge acquisition (Macků et al., 2020; Waters et al., 2018). The correlation between subjective and objective measures of knowledge varies across studies, with some showing no correlation (Ellen, 1994), while others report low to moderate correlations (Aqueveque, 2018; Han, 2019; Pieniak et al., 2010) or strong correlations (Gámbaro et al., 2013). Similarly, studies highlight diverse effects of subjective and objective knowledge on behavior (Han, 2019; Ienna et al., 2022; Kim et al., 2018; Redman & Redman, 2016). However, the majority of research on this topic originates from health and education disciplines, leaving a gap in understanding within agricultural extension programs. Evaluating these measurement approaches specifically within agricultural contexts could yield valuable insights, particularly since current evaluations of extension programs predominantly rely on subjective assessments. Theoretical and Conceptual Framework Program evaluation uses social science research methods to assess the efficacy of social intervention programs (Rossi et al., 2004). This study adapts the targeting outcomes of programs (TOP) model (Rockwell & Bennett, 2004) as its conceptual framework. The TOP Model offers a valuable framework for assessing extension program outcomes by outlining a hierarchy of effects. It posits that changes in knowledge, attitudes, skills, and aspirations (KASAs) – considered short-term outcomes – can lead to behavioral and practice changes (medium-term outcomes), ultimately contributing to improvements in social, environmental, and economic conditions (long-term outcomes or impacts). The TOP model also contributes to two purposes of evaluation, (a) to evaluate the programming process for improvement and (b) to evaluate the program outcomes for accountability. Measuring the achievement of these outcomes requires indicators, which can be subjective or objective. For instance, structured observation of participants regarding the adoption of behavior is an objective indicator for practices, while using self-reported tests is a subjective measure. Similarly, for measuring knowledge, test scores are objective indicators, and self-assessment is a subjective indicator. https://doi.org/10.37433/aad.v5i4.519 Joshi et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i4.519 61 Additionally, behavior change theories, such as the theory of planned behavior (Ajzen, 1991), social cognitive theory (Bandura, 1977), and the knowledge-attitude-behavior model (Bettinghaus, 1986; Liu et al., 2016), explain the impact of perceived knowledge (or cognition) on behavior. These theories emphasize that attitudes, beliefs, and an individual's subjective or perceived understanding significantly impact behavior change. Several studies reveal a stronger effect of objective knowledge on an individual's behavior (Ienna et al., 2022; Klerck & Sweeney, 2007; Sharma et al., 2008). The use of both subjective and objective outcome indicators would complementarily allow for triangulation of the findings, rigorously making the outcome evaluations more realistic and useful. However, extension educators lack resources, time, and skill to conduct comprehensive evaluations (Diaz et al., 2019; Kluchinski, 2014; Kumar Chaudhary et al., 2020) and are not able to use both types of indicators. Therefore, in this study, we assess whether subjective or objective indicators are the best measures of knowledge change and their relationship with behavior/practice change. Purpose The purpose of this research was to compare subjective and objective assessment measures used in agricultural extension programs, to identify the most effective approach for accurately evaluating changes in knowledge and subsequent behavior. To guide this study, we used the following objectives: 1. Determine correlations between participants' subjective (perceived) and objective (tested) knowledge in an extension program. 2. Compare the efficacy of subjective versus objective knowledge measures at predicting engagement in behavior following extension program participation. Methods We conducted a quasi-experimental, longitudinal quantitative survey (Creswell & Creswell, 2017) using Google Forms™ to collect data from Master Gardener Volunteer (MGV) training attendees in Leon (n = 15), Orange (n = 16), Osceola (n = 10), and Palm Beach (n = 16) counties in Florida during spring 2023. Data were collected before (n = 58) and after (n = 41) the 50-hour training program. The pretest questionnaire included 122 questions on subjective knowledge, objective knowledge, and engagement in gardening practices covering topics like Florida friendly landscaping, botany, entomology, plant pathology, nematology, soil, nutrients, fertilizers, turfgrass, landscape plant selection and maintenance, pesticides, vegetable gardening, fruits, propagation, and wildlife. The post-test questionnaire included additional questions on retrospective subjective knowledge, i.e., participants were asked to reflect on their knowledge before the training. This was done to identify any differences in knowledge assessment by the participants. The question item was ‘How would you rate your knowledge in the following areas prior to the Master Gardener Training Program?’ Subjective knowledge was assessed through thirteen different items using a five-point Likert scale (0 = not knowledgeable at all to 4 = extremely knowledgeable) (e.g., How would you rate https://doi.org/10.37433/aad.v5i4.519 Joshi et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i4.519 62 your knowledge in the following areas – ‘Plant pathology: diseases affecting plants’). Objective knowledge was assessed through multiple-choice questions with a correct answer. The maximum score that one could get on objective questions was 118. Engagement in gardening practices was evaluated through eleven different items (e.g., Select the response that best reflects the frequency for which you are using the practice – ‘I use irrigation as a supplement to rain’). Engagement was measured on a five-point Likert scale, based on the transtheoretical model for behavior change (Prochaska & Velicer, 1997), which was later condensed to four points for consistency in both pre and post-tests (0 = not important to me to 3 = I am doing this most of the time/all the time). Indices were created by averaging all items under each construct. Post-hoc reliability analyses for internal consistency for Likert scale questions were conducted, which resulted in Cronbach’s alpha, α >.85 for subjective knowledge question and α >.79 for behavior question. Since all the indices exceeded the minimum threshold of .7 for Cronbach’s alpha measures were considered reliable for use (Nunnally & Bernstein, 1994). A panel of experts in program evaluation, survey methodology, and MGV training were invited to review the instruments and provide feedback to ensure content and face validity. The research team revised the instruments accordingly and resubmitted them for further review. After reaching a consensus, we conducted cognitive interviews with existing MGVs to ensure the instrument's clarity for the target audience (Kumar Chaudhary & Israel, 2014). Data Cleaning and Analysis After preliminary data cleaning, we found that only 34 response pairs matched in both pre-and post-test assessments; all other cases were discarded. Upon further cleaning through regression diagnostics, we identified two multivariate outliers using Mahalanobis distance (p < 0.001), which were removed. Standardized, studentized, and studentized deleted residuals were checked for influential cases, but none were found (all residuals within ±3 SD). For research objective two, the regression assumptions were tested, and all were met. A total of 32 responses were analyzed using IBM SPSS (Version 29.0). We used descriptive statistics and paired t-tests for research objective one and Pearson correlation and hierarchical linear regression for research objective two. The effect size measure for paired t-test, i.e., Cohen’s d was interpreted using Cohen’s (1988) guidelines, .2, .5., and .8 as small, medium, and large effect size, respectively. The Pearson correlation effect size measure, i.e., correlation coefficient (Pearson’s r), was interpreted using Davis’s (1971) guidelines, .01 to .09, .1 to .29, .3 to .49, .5 to .69, and .7 and higher as negligible, association, low association, moderate association, substantial association, and very strong association, respectively. Finally, R2 was used as an effect size measure for hierarchical linear regression (Kotrlik et al., 2011). For hierarchical linear regression, engagement in gardening practices after the training was the dependent variable, and independent variables entered in the first block (model 1) were objective scores in the pre and post-test, and variables entered in the second block (model 2) were subjective knowledge scores pre and post-test. https://doi.org/10.37433/aad.v5i4.519 Joshi et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i4.519 63 Findings We found that the subjective knowledge index increased from 0.92 before the training to 2.41 after the training (see Table 1). Similarly, objective scores increased from 64.47 before the training to 109.84 after the training. A slight change in engagement in gardening practices was found from 2.40 to 2.61 before and after training, respectively. The paired t-test indicated a significant change in subjective knowledge, objective knowledge, and engagement in gardening practices before and after the training (see Table 2). We found a small effect size (d = 0.28) for the change in mean subjective knowledge from the pre-test to the post-test. There was a medium effect size for the change in engagement in gardening (d = 0.55) before and after the training. Retrospective subjective knowledge showed a large effect size (d = 2.76), indicating a substantial change from pretest to posttest. Similarly, objective knowledge also exhibited a large effect size (d = 2.83) before and after the training. Table 1 Summary of Subjective Knowledge, Objective Knowledge, and Engagement in Gardening Practices Before and After the Spring 2023 MGV Training for Participants (n = 32) Variable M SD Index of subjective knowledge before training 0.92 0.39 Index of subjective knowledge after training 2.41 0.49 Index of subjective knowledge prior training (retrospective pretest) 0.94 0.45 Objective score before training 64.47 15.04 Objective score after training 109.84 14.56 Index of engagement in gardening practices before training 2.41 0.57 Index of engagement in gardening practices after training 2.61 0.41 Note. The response scale for subjective knowledge was: Not knowledgeable at all (0), Slightly knowledgeable (1), Moderately knowledgeable (2), Very knowledgeable (3), and Extremely knowledgeable (4). Response scales for engagement in gardening practices were: Not important to me (0), Considering this (1), I'm doing this occasionally (2), and I am doing this most of the time/all the time (3). https://doi.org/10.37433/aad.v5i4.519 Joshi et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i4.519 64 Table 2 Results of Paired t-Tests for Subjective Knowledge, Objective Knowledge, and Engagement in Gardening Practices for Spring 2023 MGV Training Participants (n = 32) Variable Mean before Mean after Paired mean difference Confidence interval of the difference p-value Cohen’s d Lower Upper Subjective knowledge Pretest – posttest 0.92 2.41 1.49** 1.304 1.686 <0.001 0.28 Retrospective pretest – posttest 0.94 2.41 1.47** 1.282 1.666 <0.001 2.76 Objective knowledge Pretest – posttest 64.72 109.84 45.12** 39.380 50.869 <0.001 2.83 Engagement in gardening practices Pretest – posttest 2.41 2.61 0.21** 0.075 0.349 0.004 0.55 *indicates significance at p ≤ 0.05; **indicates significance at p ≤ 0.01 Objective One: Relationships Between Subjective and Objective Knowledge Assessments We found a significant correlation between subjective and objective knowledge after training with moderate association (r = 0.398, p = 0.024) (see Table 3). A very strong association (r = 0.720, p < 0.001) between retrospective and baseline pretest was also found. No significant correlation was found between subjective and objective knowledge before the training, retrospective subjective knowledge and objective knowledge before training, and pretest subjective and posttest objective knowledge. Table 3 Pearson Correlation Coefficients Between Subjective and Objective Knowledge at Different Training Stages for Spring 2023 MGV Training Participants (n = 32) Correlation between Item 1 Item 2 Pearson correlation p- value Subjective knowledge after the training Objective knowledge after the training 0.398* 0.024 Subjective knowledge before training Objective knowledge before training 0.087 0.637 Retrospective pretest subjective knowledge Pre-test subjective knowledge 0.720** <0.001 Retrospective pretest subjective knowledge Objective knowledge before training 0.159 0.386 Pretest subjective knowledge Post-test objective knowledge -0.025 0.892 *indicates significance at p ≤ 0.05; **indicates significance at p ≤ 0.01 https://doi.org/10.37433/aad.v5i4.519 Joshi et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i4.519 65 Linear regression was carried out to quantify the relationship between subjective and objective knowledge after training. We aimed to understand how subjective knowledge (independent variable) predicted objective knowledge (dependent variable) after the training. The result of linear regression indicated that subjective knowledge significantly predicts objective knowledge after the training (F (1, 30) = 5.654, p = 0.024) and explains 13.1% variability in objective knowledge. The unstandardized regression coefficient (B) of 11.659 indicated that every one- unit increase in subjective knowledge after training increases objective knowledge by 11.659 units. Objective Two: Compare the Efficacy of Subjective Versus Objective Knowledge Measures at Predicting Engagement in Behavior Following Extension Program Participation. The hierarchical linear regression analysis where engagement in gardening practices after training was regressed against subjective and objective knowledge before and after training (see Tables 4 & 5) indicates that only subjective knowledge before the training (B = 0.404, t = 2.284, p = 0.031) significantly predicts engagement in gardening practices after training. Model 1 which included objective knowledge before and after training was not significant (F (2, 29) = 1.904, p = 0.167). When subjective knowledge before and after training was added, i.e., Model 2, explained variability increased to 22% (adjusted R2) and the model was significant (F (2, 27) = 4.475, p = 0.021). The unstandardized regression coefficient (B) of 0.404 indicated that every one-unit increase in subjective knowledge before training increases engagement in gardening practices after training by 0.404 units when other variables were held constant. Table 4 Model Summary of Hierarchical Linear Regression of Engagement in Gardening Practices as Dependent Variable and Subjective and Objective Knowledge Before and After Training as Independent Variables Std. error of the estimate Change statistics Model R R2 Adjusted R2 R2 change F change df1 df2 Sig. f change 1 0.341 0.116 0.055 0.40117 0.116 1.904 2 29 0.167 2 0.580 0.336 0.238 0.36030 0.220 4.475 2 27 0.021 Note. Model 1 = Objective knowledge before and after training; Model 2 = Model 1 + Subjective knowledge before and after training https://doi.org/10.37433/aad.v5i4.519 Joshi et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i4.519 66 Table 5 Hierarchical Regression Analysis of Engagement in Gardening Practices After Training as Dependent Variable and Subjective and Objective Knowledge Before and After Training as Independent Variables Variable Model 1 Model 2 B β t p B β t p Objective score after training 0.008 0.276 1.436 0.162 0.006 0.227 1.177 0.249 Objective score before training 0.003 0.114 0.593 0.558 0.002 0.085 0.486 0.631 Subjective knowledge index after training 0.148 0.178 0.963 0.344 Subjective knowledge index before training* 0.404 0.387 2.282 0.031 *indicates significance at p ≤ 0.05; **indicates significance at p ≤ 0.01 Discussion, Conclusions, and Recommendations Our study sheds light on the intricate interplay between subjective and objective knowledge, their influence on engagement, and the enduring impact of pre-existing perceptions on post- training behaviors. We found a moderate correlation between subjective and objective knowledge after training, but no correlation was found before the training. Moreover, our study revealed that subjective knowledge before training is the sole predictor of post-training gardening behaviors. The lack of correlation between subjective and objective knowledge before training might be because of a lack of meta-cognitive skills among participants to access their knowledge accurately (Kruger & Dunning, 1999) or because of overprediction of their subjective knowledge (Karaca et al., 2023). Participants’ subjective and objective knowledge changed significantly after the training, leading to a more congruent alignment between the two after completing the training. Regarding behavior, i.e., engagement in gardening practices, we found an overall increase in it after the training. However, participants' original perceptions predicted this behavior, overshadowing the effect of newly acquired objective knowledge. Ajzen et al. (2011) found a similar result and described that factual information alone seldom directly translates into decision-making or actions; instead, self-appraisals shape behavioral beliefs. Similarly, the stronger impact of perceived knowledge on behavior is reported in various contexts (Han, 2019; Pieniak et al., 2010; Redman & Redman, 2016). Additionally, this finding supports the idea that altering participants’ subjective knowledge is more challenging than enhancing objective knowledge (Lonka et al., 1996). Consequently, addressing pre-existing perceptions and beliefs should be a central focus of extension programs seeking to facilitate enduring behavior change. There is a practical application of the insights derived from the findings of this study to the design and implementation of agricultural development programs and extension initiatives aimed at fostering behavior change. This is the major implication of this study contributing to the field of Extension and agricultural development. Understanding the complexities of https://doi.org/10.37433/aad.v5i4.519 Joshi et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i4.519 67 cognitive processes and the critical role of participant’s perceptions is essential for creating lasting behavioral shifts toward the planned direction in agricultural extension and development programs. By prioritizing the reshaping of perceptions and beliefs before engaging participants in training, extension educators can enhance the likelihood that newly acquired objective knowledge will transform effectively into desired agricultural behaviors. Our findings reveal a crucial area of focus for agricultural extension programs: facilitating participants to appraise more accurate self-assessments before training begins (e.g., doing reflective activities on their beliefs and assumptions), rather than solely concentrating on the delivery of information. This strategy could significantly improve the alignment between participant’s expectations and current level of knowledge and skills with the introduction of new knowledge and skills. Furthermore, our results highlight the value of integrating both subjective and objective assessments into extension program evaluations to triangulate the program outcomes realistically. This approach allows for a more comprehensive analysis of knowledge gains and better captures the perceptions most directly linked to behavior change, which is especially pertinent in the context of Extension education and agricultural development. Finally, our regression analyses shed light on the intricate relationships between subjective and objective knowledge and their impact on gardening practices within the agricultural sector. Recognizing these dynamics, extension training programs could benefit from placing greater emphasis on specific knowledge domains that have shown a more substantial influence on the application of new knowledge and skills following the training. By tailoring educational content to address these influential factors, extension programs can significantly enhance the overall efficacy of Master Gardener Volunteer (MGV) training initiatives and other agricultural education efforts, ultimately aiding in the effective dissemination of research-based horticultural information and supporting the broader goals of sustainable agricultural development. Limitations To address the limitations of this study, particularly the small sample size and one group of extension audience, future research should replicate this study with a diverse and larger participant base. Additionally, it is advisable to implement strategies such as targeted follow-up communication, incentives, or other engagement measures to encourage post-training application, which is critical for sustaining agricultural practices introduced through extension programs. In terms of questionnaire design, a more concise and streamlined set of questions is recommended to improve response rates, reduce survey fatigue, and ensure efficient data collection, thereby enhancing the effectiveness of agricultural education evaluations. https://doi.org/10.37433/aad.v5i4.519 Joshi et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i4.519 68 Acknowledgments This work was supported by funding from the Penn State Extension’s Multistate & Integrated Grants Program. This work was partially supported by the USDA National Institute of Food and Agriculture and Hatch Appropriations under Project #PEN04945 and Accession #7006518 titled “Applying Socio-psychological Factors to Enhance Educational Programs and Policies.” Author Contributions: A. Joshi – Data curation, formal analysis, writing-original draft, writing- reviewing & editing; J. Diaz – Conceptualization, methodology, validation, formal analysis, investigation, writing-reviewing & editing, supervision, funding acquisition; A. Kumar Chaudhary – Conceptualization, methodology, validation, formal analysis, investigation, writing-reviewing & editing, supervision, funding acquisition; K. S. U. Jayaratne – Conceptualization, methodology, validation, formal analysis, investigation, writing-reviewing & editing, supervision, funding acquisition; S. Galindo – Writing-reviewing & editing, supervision. References Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211. https://doi.org/10.1016/0749-5978(91)90020-T Ajzen, I., Joyce, N., Sheikh, S., & Cote, N. G. (2011). Knowledge and the prediction of behavior: The role of information accuracy in the theory of planned behavior. Basic and Applied Social Psychology, 33(2), 101–117. https://doi.org/10.1080/01973533.2011.568834 Aqueveque, C. (2018). Ignorant experts and erudite novices: Exploring the Dunning-Kruger effect in wine consumers. Food Quality and Preference, 65, 181–184. https://doi.org/10.1016/j.foodqual.2017.12.007 Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191–215. https://doi.org/10.1037/0033-295X.84.2.191 Bettinghaus, E. P. (1986). Health promotion and the knowledge-attitude-behavior continuum. Preventive Medicine, 15(5), 475–491. https://doi.org/10.1016/0091-7435(86)90025-3 Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates. Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). SAGE Publications. Davis, J. A. (1971). Elementary survey analysis. Prentice-Hall. https://doi.org/10.37433/aad.v5i4.519 https://doi.org/10.1016/0749-5978(91)90020-T https://doi.org/10.1080/01973533.2011.568834 https://doi.org/10.1016/j.foodqual.2017.12.007 https://psycnet.apa.org/doi/10.1037/0033-295X.84.2.191 https://doi.org/10.1016/0091-7435(86)90025-3 Joshi et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i4.519 69 Diaz, J. M., Jayaratne, K. S. U., & Kumar Chaudhary, A. (2019). Evaluation competencies and challenges faced by early career extension professionals: Developing a competency model through consensus building. The Journal of Agricultural Education and Extension, 26(2), 1–19. https://doi.org/10.1080/1389224x.2019.1671204 Ellen, P. S. (1994). Do we know what we need to know? Objective and subjective knowledge effects on pro-ecological behaviors. Journal of Business Research, 30(1), 43–52. https://doi.org/10.1016/0148-2963(94)90067-1 Gámbaro, A., Ellis, A. C., & Prieto, V. (2013). Influence of subjective knowledge, objective knowledge and health consciousness on Olive oil consumption—A case study. Food and Nutrition Sciences, 4(04), 445–453. https://doi.org/10.4236/fns.2013.44057 Gonyea, R. M. (2005). Self-reported data in institutional research: Review and recommendations. New Directions for Institutional Research, 2005(127), 73–89. https://doi.org/10.1002/ir.156 Han, T. I. (2019). Objective knowledge, subjective knowledge, and prior experience of organic cotton apparel. Fashion and Textiles, 6(1), 4. https://doi.org/10.1186/s40691-018-0168- 7 House, L. O., Lusk, J., Jaeger, S. R., Traill, B., Moore, M., Valli, C., Morrow, B., & Yee, W. (2004, August 1-4). Objective and subjective knowledge: Impacts on consumer demand for genetically modified foods in the United States and the European Union. 2004 American Agricultural Economics Association Annual Meeting, Denver, CO, USA. https://agbioforum.org/wp-content/uploads/2021/02/AgBioForum_7_3_113.pdf Ienna, M., Rofe, A., Gendi, M., Douglas, H. E., Kelly, M., Hayward, M. W., Callen, A., Klop-Toker, K., Scanlon, R. J., Howell, L. G., & Griffin, A. S. (2022). The relative role of knowledge and empathy in predicting pro-environmental attitudes and behavior. Sustainability, 14(8), 4622. https://doi.org/10.3390/su14084622 Karaca, M., Geraci, L., Kurpad, N., Lithander, M. P., & Balsis, S. (2023). Low-performing students confidently overpredict their grade performance throughout the semester. Journal of Intelligence, 11(10), 188. https://doi.org/10.3390/jintelligence11100188 Kim, M. S., Kim, J., & Thapa, B. (2018). Influence of environmental knowledge on affect, nature affiliation and pro-environmental behaviors among tourists. Sustainability, 10(9), 3109. https://doi.org/10.3390/su10093109 Klerck, D., & Sweeney, J. C. (2007). The effect of knowledge types on consumer-perceived risk and adoption of genetically modified foods. Psychology & Marketing, 24(2), 171–193. https://doi.org/10.1002/mar.20157 https://doi.org/10.37433/aad.v5i4.519 https://doi.org/10.1080/1389224x.2019.1671204 https://doi.org/10.1016/0148-2963(94)90067-1 https://doi.org/10.4236/fns.2013.44057 https://doi.org/10.1002/ir.156 https://doi.org/10.1186/s40691-018-0168-7 https://doi.org/10.1186/s40691-018-0168-7 https://agbioforum.org/wp-content/uploads/2021/02/AgBioForum_7_3_113.pdf https://doi.org/10.3390/su14084622 https://doi.org/10.3390/jintelligence11100188 https://doi.org/10.3390/su10093109 https://doi.org/10.1002/mar.20157 Joshi et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i4.519 70 Kluchinski, D. (2014). Evaluation behaviors, skills and needs of cooperative extension agricultural and resource management field faculty and staff in New Jersey. Journal of the NACAA, 7(1). https://www.nacaa.com/journal/c5c991c9-960b-4187-871a- 3e027abaccdd Kotrlik, J. W., Williams, H. A., & Jabor, M. K. (2011). Reporting and interpreting effect size in quantitative agricultural education research. Journal of Agricultural Education, 52(1), 132-142. https://doi.org/10.5032/jae.2011.01132 Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121. https://doi.org/10.1037/0022-3514.77.6.1121 Kumar Chaudhary, A., Diaz, J., Jayaratne, K. S. U., & Assan, E. (2020). Evaluation capacity building in the nonformal education context: Challenges and strategies. Evaluation and Program Planning, 79, 101768. https://doi.org/10.1016/j.evalprogplan.2019.101768 Kumar Chaudhary A., & Israel G. D. (2014, December). The savvy survey # 8: Pilot testing and pretesting methods of pretesting (AEC402). Agricultural Education and Communication Department, UF/IFAS Extension. http://edis.ifas.ufl.edu/publication/PD072 Larese-Casanova, M. (2017). Measuring program impacts: 4. Evaluating long-term impacts. Utah State University Extension. https://digitalcommons.usu.edu/cgi/viewcontent.cgi?article=2679&context=extension_ curall Liu, L., Liu, Y. P., Wang, J., An, L. W., & Jiao, J. M. (2016). Use of a knowledge-attitude-behaviour education programme for Chinese adults undergoing maintenance haemodialysis: Randomized controlled trial. Journal of International Medical Research, 44(3), 557–568. https://doi.org/10.1177/0300060515604980 Lonka, K., Joram, E., & Bryson, M. (1996). Conceptions of learning and knowledge: Does training make a difference? Contemporary Educational Psychology, 21(3), 240–260. https://doi.org/10.1006/ceps.1996.0021 Macků, K., Caha, J., Pászto, V., & Tuček, P. (2020). Subjective or objective? How objective measures relate to subjective life satisfaction in Europe. ISPRS International Journal of Geo-Information, 9(5), 320. https://doi.org/10.3390/ijgi9050320 Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill. O’Leary, J., & Israel, G. (2019). Capturing change: Comparing pretest-posttest and retrospective evaluation methods. University of Florida IFAS Extension, WC135. https://edis.ifas.ufl.edu/publication/WC135 https://doi.org/10.37433/aad.v5i4.519 https://www.nacaa.com/journal/c5c991c9-960b-4187-871a-3e027abaccdd https://www.nacaa.com/journal/c5c991c9-960b-4187-871a-3e027abaccdd https://doi.org/10.5032/jae.2011.01132 https://doi.org/10.1037/0022-3514.77.6.1121 https://doi.org/10.1016/j.evalprogplan.2019.101768 http://edis.ifas.ufl.edu/publication/PD072 https://digitalcommons.usu.edu/cgi/viewcontent.cgi?article=2679&context=extension_curall https://digitalcommons.usu.edu/cgi/viewcontent.cgi?article=2679&context=extension_curall https://doi.org/10.1177/0300060515604980 https://doi.org/10.1006/ceps.1996.0021 https://doi.org/10.3390/ijgi9050320 https://edis.ifas.ufl.edu/publication/WC135 Joshi et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v5i4.519 71 Pieniak, Z., Aertsens, J., & Verbeke, W. (2010). Subjective and objective knowledge as determinants of organic vegetables consumption. Food Quality and Preference, 21(6), 581-588. https://doi.org/10.1016/j.foodqual.2010.03.004 Prochaska, J. O., & Velicer, W. F. (1997). The transtheoretical model of health behavior change. American Journal of Health Promotion, 12(1), 38–48. https://doi.org/10.4278/0890- 1171-12.1.38 Redman, A., & Redman, E. (2016). Is subjective knowledge the key to fostering sustainable behavior? Mixed evidence from an education intervention in Mexico. Education Sciences, 7(1), 4. https://doi.org/10.3390/educsci7010004 Rockwell, K., & Bennett, C. (2004). Targeting outcomes of programs: A hierarchy for targeting outcomes and evaluating their achievement. Department of Agricultural Leadership, Education, and Communication: Faculty Publications. http://digitalcommons.unl.edu/aglecfacpub/48/ Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: A systematic approach. SAGE Publications. Sharma, S. V., Gernand, A. D., & Day, R. S. (2008). Nutrition knowledge predicts eating behavior of all food groups except fruits and vegetables among adults in the Paso del Norte region: Qué Sabrosa Vida. Journal of Nutrition Education and Behavior, 40(6), 361–368. https://doi.org/10.1016/j.jneb.2008.01.004 Waters, E. A., Biddle, C., Kaphingst, K. A., Schofield, E., Kiviniemi, M. T., Orom, H., Li, Y., & Hay, J. L. (2018). Examining the interrelations among objective and subjective health literacy and numeracy and their associations with health knowledge. Journal of General Internal Medicine, 33(11), 1945–1953. https://doi.org/10.1007/s11606-018-4624-2 © 2024 by authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/). https://doi.org/10.37433/aad.v5i4.519 https://doi.org/10.1016/j.foodqual.2010.03.004 https://doi.org/10.4278/0890-1171-12.1.38 https://doi.org/10.4278/0890-1171-12.1.38 https://doi.org/10.3390/educsci7010004 http://digitalcommons.unl.edu/aglecfacpub/48/ https://doi.org/10.1016/j.jneb.2008.01.004 https://doi.org/10.1007/s11606-018-4624-2