
































Microsoft Word - 91-Manuscript-1445-1-11-20210712.docx


Street et al.  Advancements in Agricultural Development 
  Volume 2, Issue 2, 2021 
  agdevresearch.org 

1. Ashton Street, Agriculture Teacher, Harmony High School, 3601 Arthur J Gallagher Blvd., Harmony, FL 34771, 

ashton.street@osceolaschools.net, https://orcid.org/0000--0002-4900-0824 
2. Christopher T. Stripling, Associate Professor & Interim Department Head, University of Tennessee, 320 Morgan Hall, 2621 

Morgan Circle Drive, Knoxville, TN 379966, cstripli@utk.edu, https://orcid.org/0000--0002-5045-3492 
3. John C. Ricketts, Professor, Tennessee State University, 3500 John A. Merritt Boulevard, Nashville, TN 37209, 

jricket1@tnstate.edu, https://orcid.org/0000--0002-0559-5803 
4. Nathan W. Conner, Associate Professor, University of Nebraska-Lincoln, 236 Filley Hall, Lincoln, NE 68583, 

nconner2@unl.edu, https://orcid.org/0000--0003-0063-4110 
5. Christopher Boyer, Associate Professor, University of Tennessee, 320 Morgan Hall, 2621 Morgan Circle Drive, Knoxville, TN 

37996, cboyer3@utk.edu, https://orcid.org/0000--0002-1393-8589 
 

86 

 

Identifying Tennessee School-based Agricultural Education 
Student Growth and Program Accountability Metrics 

 
A. Street1, C. Stripling2, J. Ricketts3, N. Conner4, C. Boyer5 

 
 

 

  

Abstract 

Over the years, accountability in education has transformed from the primary focus being the school 
as a whole to the individual teacher. The purpose of this study was to determine the metrics 
Tennessee school-based agricultural education teachers perceive as indicators of excellent total 
programs (classroom instruction, FFA, SAE), and a modified Delphi study was used to seek a 
consensus. The following nine metrics were retained: (a) pesticide certification, (b) program of 
activities, (c) number of students participating in CDEs, (d) chapter community service hours, (e) total 
number of FFA activities, (f) number of CDEs coached, (g) at least one proficiency at regional level, (h) 
one American degree every 3 years, and (i) percentage of students with SAE. Overall, the metrics 
agreed upon are narrow in focus and all but one is a record of activity and not direct measures of 
students’ knowledge or skills. As a result, the measures do not include student growth or value-
added scores or authentic assessments of 21st century skills. Additional research is needed to further 
investigate the metrics that should be used to measure a school-based agricultural education 
program’s success in Tennessee and across the nation. 
 
 

Keywords 

Delphi method, total program, performance evaluation, SBAE 



Street et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v2i2.91   87 
 

Introduction and Problem Statement 
 
Accountability in education transformed from the primary focus being the whole school to the 
individual teacher (Lavigne, 2014). A focus on teachers being accountable for student success 
has altered education in the United States (Buchanan, 2014). According to the Reform Support 
Network (2013), value-added measures “convey how much individual teachers contribute to 
student learning in a particular subject in a particular year” ( p. 2). However, researchers and 
policymakers found value-added measures alone are not an adequate replacement for 
traditional teacher evaluation (Harris et al., 2014). 
 
Tennessee has been a frontrunner for value-added assessment and was one of the first states 
to apply a statewide evaluation system (Lavigne, 2014). In the 2011-2012 school year, the 
Tennessee Educator Acceleration Model (TEAM) was implemented statewide (Lavigne, 2014). 
TEAM outlines that 50% of a teacher’s evaluation is comprised of student achievement data 
and within that percentage, 35% is calculated from student growth using students’ value-added 
scores, and the other 15% is made up of other student achievement measures as decided by 
the State Board of Education and agreed upon by the teacher and the evaluator (Lavigne, 
2014). The other 50% is comprised of personal conferences, observation data, and a review of 
prior teaching evaluations (Lavigne, 2014). Glover et al. (2016) recommended using multiple 
types of formative assessment techniques for teacher and student evaluation.  
 
School-based agricultural education (SBAE) is one of the disciplines included in Career and 
Technical Education (CTE). CTE programs are heavily influenced by the Carl D. Perkins 
legislation, which was designed to enhance the academic and technical skill development of 
secondary CTE students (Johnson & Mitchell, 2014). The core indicators for secondary CTE 
programs focus on academics, technical skills, graduation, post-high school education, and 
career trajectory (Johnson & Mitchell, 2014). Due to the emphasis on value-added measures in 
Tennessee and the need to consider core indicators of CTE programs, this study seeks to 
capture the voice of Tennessee SBAE teachers regarding the metrics they perceive the 
Tennessee Department of Education (TDE) should use to measure student growth and total 
SBAE program success. 
 

Theoretical and Conceptual Framework 
 
SBAE programs are designed for students to study and investigate solutions for real problems in 
the agriculture industry; therefore, the content being studied is flexible and changing (Phipps et 
al., 2008). For this study, SBAE’s three-component model served as the conceptual framework. 
The model includes three main instructional components: (a) classroom/laboratory instruction, 
(b) student leadership development through the National FFA Organization, and (c) supervised 
agricultural experience (SAE) programs (National FFA Organization, 2019). 
 
Classroom/laboratory instruction provides students with the opportunity to discuss and study 
current and future challenges that pertain to particular areas of the agriculture industry, and 



Street et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v2i2.91   88 
 

“as a result, students gain knowledge and discover principles that allow them to arrive at 
potential solutions to agricultural problems” (Phipps et al., 2008, p. 5.) Through participation in 
the National FFA Organization, “students develop excellent skills in leading and working with 
people of all ages” (Phipps et al., 2008, p. 7). The FFA is an intra-curricular program that offers 
students challenging activities and programs for both the individual student and the whole 
school chapter (Phipps et al., 2008). The final component, SAE, provides students with the 
opportunity to apply the knowledge and skills they learn within school to real-world situations 
through their own SAE program outside of the classroom (Phipps et al., 2008). Students will 
work on this program throughout their high school career and gain experiences in a real-life 
setting within an agriculture industry (Phipps et al., 2008). Example SAE’s include livestock 
production and working in an agricultural business (Phipps et al., 2008). Each component of the 
three-component model was used to develop an open-ended question that would be used in 
round one of this study. 
 

Purpose 
 
The purpose of this study was to determine the metrics Tennessee SBAE teachers perceived as 
indicators of excellent total programs. Additionally, the following research question guided this 
study: What metrics should TDE use to measure SBAE student growth and the total program?  
 

Methods 
 
A modified Delphi study was used to seek a consensus on metrics SBAE teachers perceived as 
indicators of excellent total programs. The modified Delphi method has been frequently used 
by researchers in agricultural education (Lundry et al., 2015; Terry & Osborne, 2015). Three 
rounds of data collection were used to solicit the opinions of an expert panel of SBAE teachers. 
The criteria for membership on the panel were twofold: (a) currently working as a teacher in 
SBAE in Tennessee and (b) nominated by members of Tennessee team agricultural education. 
Team agricultural education is an advisory committee for TDE consisting of teacher educators, 
SBAE teachers, a community college agriculture instructor, a National FFA staff member, and 
representatives of the Tennessee Association of Agricultural Educators. Twenty-two teachers 
were nominated, and 21 teachers were invited to participate in this study. We decided to invite 
seven teachers from each of the three regions of Tennessee and eliminate the teacher with the 
fewest years of experience.  
 
An email was sent to the 21 teachers informing them of the study, their selection, and the study 
procedures. The data collection process consisted of three rounds. Qualtrics was used to 
distribute a survey for each round in order to reach a general consensus.  
 
Round one consisted of the following open-ended prompts that were developed based on the 
three-component model of agricultural education: (a) Please list all of the possible agriculture 
classroom measures (i.e., TEAM scores, end of course test, certifications) that you think ought 
to be used to provide student achievement/program data to TDE; (b) Please list all of the 



Street et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v2i2.91   89 
 

possible FFA measures (i.e., % FFA membership, CDE participation) that you think ought to be 
used to provide student achievement/program data to TDE; and (c) Please list all of the possible 
SAE measures (i.e., percentage with an SAE, dollars earned or hours worked) that you think 
ought to be used to provide student achievement/program data to TDE. After the initial survey 
was distributed and two reminders were sent, we obtained responses from 17 of the 21 
teachers (81.0%) in round one.  
 
A five-point rating scale (1= strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = 
agree, and 5 = strongly agree) was used in round two and allowed the teachers to rate their 
level of agreement or disagreement for each of the 59 items. We established a priori a 66.7% or 
2/3 level of agreement for an item to be retained for round three based on prior works (Shinn 
et al., 2008; Touchstone, 2015). In addition to the 59 items identified in round one, we included 
the following open-ended prompt in round two: Please list additional measures you believe 
should be included that were not included in Round 2. The additional open-ended prompt was 
asked to give participants the opportunity to share any additional measures that came to mind 
during round two as a result of seeing the measures identified in round one. We used an open-
ended prompt in round two based on prior research (Conner et al., 2017; Warner et al., 2016). 
After three reminders to complete round two, 18 of the 21 teachers (85.7%) responded.  
 
Therefore, the third and final round was comprised of 19 items that were rated by the teachers 
using the five-point rating scale. As in round two, a 66.7% or 2/3 level of agreement was used to 
retain an item as a possible metric for measuring a component of the total SBAE program. In 
round three, responses were obtained from 19 of the 21 teachers (90.5%). 
 

Findings 
 
The following categories are used to group together the findings: classroom, FFA, and SAE. 
Round 1 generated 59 unique responses from the expert panel of teachers (see Table 1). 
Fourteen items met the predetermined 66.7% level of agreement.  
 
  



Street et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v2i2.91   90 
 

Table 1 
 
Metrics Identified as Indicators of Excellent Agricultural Education Total Programs 
Metrics Agree/Strongly Agree % 

1. Classroom  
2. Master Livestock certification (beef, meat, goat, advanced, 

etc.) 72.3 a 

3. Pesticide certification 66.7 a 
4. Beef Quality Assurance certification 66.7 a 
5. Hunter Education certification 66.7 a 
6. Welding certification 66.7 a 
7. Number of students who enroll in post-secondary studies 66.7 a 
8. Master Gardner certification 55.6 
9. Number of students who enroll in postsecondary 

agricultural studies 55.6 

10. Skill attainment rubrics 55.6 
11. Tractor Safety certification 55.5 
12. Dual credit 50.0 
13. Dual enrollment 50.0 
14. End of course exam 44.5 
15. Boater Education certification 38.9 
16. Tennessee Educator Acceleration Model (TEAM) scores 22.2 

   FFA  
17. Program of activities 94.5 a 
18. Number of students participating in CDEs 77.7 a 
19. Chapter community service hours 72.2 a 
20. Average number of volunteer hours per student 66.7 a 
21. Number of CDEs coached 66.7 a 
22. Number of Team CDEs coached 55.6 
23. Number of Individual CDEs coached 55.6 
24. Number of students attending State FFA Convention 55.5 
25. Regional officers elected 52.9 
26. FFA membership percentage of agricultural education 

enrollment 50.0 

27. State officers nominated 50.0 
28. Plow awards program 50.0 
29. Regional officers nominated 50.0 
30. State officers elected 44.4 
31. Number of students 38.9 
32. Average number of students attending local meetings 38.9 
33. Number of students receiving FFA scholarships 38.9 
34. Number of students attending 212/360 Conference 27.8 
35. National officer candidate 22.2 
36. National officer elected 17.6 



Street et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v2i2.91   91 
 

Metrics Agree/Strongly Agree % 
37. Number of students receiving FFA grants 11.1 
38. Number of FFA chapter grants awarded 11.1 

SAE  
39. Percentage of students with SAE 77.2 a 
40. One American degree every 3 years 76.5 a 
41. At least one proficiency at regional level 72.2 a 
42. Number of students with an SAE record book 55.5 
43. Number of State FFA degree recipients each year 55.5 
44. Average dollars earned per student 44.5 
45. Number of American degree recipients each year 44.5 
46. Average hours worked or dollars earned per student 38.9 
47. Number of proficiency applications 38.9 
48. Number of students exhibiting livestock 38.9 
49. Average dollars expended per student 38.9 
50. Average hours worked per student 35.3 
51. Number of Regional Star candidates 33.3 
52. Number of Regional Star recipients 27.8 
53. Number of TN Star candidates 27.8 
54. Number of TN Star recipients 27.8 
55. Number of National FFA star candidates 22.2 
56. Number of regional proficiencies in first place 22.0 
57. Number of regional proficiencies in second place 22.0 
58. Number of regional proficiencies in third place 22.0 
59. Number of proficiencies not in top 3 16.7 
60. Number of National FFA Star recipients 16.7 

a Retained for Round 3. 
 
Five additional metrics were obtained from Round 2. Therefore, the remaining 19 items were 
retained for Round 3 (see Table 2). The metrics added were: (a) teacher is CASE certified, (b) 
CASE online student assessments, (c) CASE authentic assessment rubrics, (d) Pork Quality 
Assurance certification, and (e) total number of FFA activities. 
 
  



Street et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v2i2.91   92 
 

Table 2 
 
Metrics Suggested in Round 2 as Indicators of Excellent Agricultural Education Programs  
Metrics Agree/Strongly Agree % 
Classroom  

1. Pesticide certification 68.4 a 
2. Welding certification 63.2 
3. Beef Quality Assurance certification 63.1 
4. Pork Quality Assurance certification 57.9 
5. Hunter Education certification 52.6 
6. Number of students who enroll in postsecondary education  52.6 
7. Master Livestock certification 52.6 
8. Teacher is CASE certified 36.8 
9. CASE online student assessments 36.8 
10. CASE authentic assessment rubrics 36.9 

FFA  
11. Program of Activities 94.7 a 
12. Number of students participating in CDEs 84.2 a 
13. Chapter community service hours 78.9 a 
14. Total number of FFA activities 78.9 a 
15. Number of CDEs coached 68.4 a 
16. Average number of volunteer hours per student 36.8 

SAE  
17. At least one proficiency at regional level 84.2 a 
18. One American Degree every 3 years 78.9 a 
19. Percentage of students with SAE 77.7 a 

a Retained as Indicators of Excellent Agricultural Education Total Programs 
 
The third and final round began with 19 metrics and resulted in the removal of 10 metrics due 
to lack of consensus: (a) Beef Quality Assurance certification, (b) Master Livestock certification, 
(c) Hunter Education certification, (d) Welding certification, (e) number of students who enroll 
in postsecondary education, (f) teacher is CASE certified, (g) CASE online student assessments, 
(h) CASE authentic assessment rubrics, (i) Pork Quality Assurance certification, and (j) average 
number of volunteer hours per student. The reduction of metrics left a general consensus that 
nine metrics (see Table 3) generated from previous rounds should be used in measuring SBAE 
total program success.  
 
  



Street et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v2i2.91   93 
 

Table 3 
 
Metrics Retained as Indicators of Excellent Agricultural Education Total Programs 
 Metrics 
Classroom  
 1. Pesticide certification 
FFA  
 2. Program of Activities 
 3. Number of students participating in CDEs 
 4. Chapter community service hours 
 5. Total number of FFA activities 
 6. Number of CDEs coached 
SAE  
 7. At least one proficiency at regional level 
 8. One American Degree every 3 years 
 9. Percentage of students with SAE 

 
Conclusions, Discussion, and Recommendations 

 
The SBAE teachers ultimately agreed upon nine metrics, one metric for classroom instruction, 
five metrics for FFA, and three metrics for SAE as measures of total program success. Overall, 
the metrics agreed upon are narrow in focus, and all but one is a record of activity and not 
direct measures of students’ knowledge or skills. As a result, the measures do not include 
student growth or value-added scores (Lavigne, 2014) or authentic assessments of 21st-century 
skills (i.e., leadership, communication, critical thinking, and teamwork). The focus on measuring 
success based on activity within FFA and SAE may be due to a culture that promotes 
competition rather than academic learning. Therefore, we recommend research be conducted 
to investigate Tennessee agriculture education teachers’ perceptions of the role of the 
classroom instructional component of the SBAE model related to the total SBAE program. These 
insights may shed light on the narrow focus of the metrics identified and the lack of consensus 
regarding classroom instruction metrics. 
 
The one metric the panel agreed upon for classroom instruction was pesticide certification. The 
core indicators for secondary education included technical skill attainment, including 
achievement on technical assessments aligned with industry-recognized standards (Johnson & 
Mitchell, 2014). Based on Johnson and Mitchell (2014), pesticide certification aligns with the 
current requirements of Perkins IV but may not be practical in nature due to the fact an 
individual must be 18 years old to receive certification. Furthermore, this metric of student 
learning is not applicable to all SBAE programs of study in Tennessee. Additional certifications 
were recognized but did not pass the consensus to be retained. This may be due to the various 
programs of study and courses offered within Tennessee. 
 



Street et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v2i2.91   94 
 

Due to the lack of agreement on numerous metrics in the classroom setting, TDE should pursue 
further consensus among SBAE educators on other classroom metrics before implementing 
statewide classroom accountability metrics. To that end, future research is needed to 
determine and, if needed, develop appropriate classroom metrics. TDE should look to other 
state departments of education, CTE and agricultural education consultants, national CTE and 
agricultural education leaders, industry partners, and SBAE teachers to further explore 
classroom metrics. We recommend metrics such as TEAM evaluations, meaningful industry 
certifications, and cognitive skills be considered. According to Lavigne (2014), the TEAM 
evaluations will take into consideration multiple data points, including student achievement 
data, personal conferences, observation data, and the review of the teacher’s prior evaluations 
(Lavigne, 2014). The collection and evaluation of teacher data will help administrators provide 
constructive feedback to educators (Firestone, 2014). The use of student achievement data 
both from an academic perspective and from industry certifications will help CTE programs 
meet the requirements set forth by the Perkins ACT (Johnson & Mitchell, 2014). In addition, 
student skill assessments appropriate for each SBAE program of study should be considered.  
 
In regards to FFA, metrics were agreed upon, which support the Perkins IV core indicator of 
having a career and technical student organization as a support for the instructional program 
(Johnson & Mitchell, 2014). The five metrics can be used as a starting point for accountability 
and should be reviewed by TDE for appropriateness. In addition, the department should review 
the purpose and goals of FFA as these metrics may not provide data on TDE’s goals for FFA.  
 
The panel identified three metrics focused on SAE measures. Percentage of students with an 
SAE supports the state’s criteria that a majority of the students enrolled in SBAE courses have in 
place an approved supervised agricultural experience program. At least one proficiency at the 
regional level is like the state’s current criteria of submitting one or more proficiency awards for 
state consideration. However, the state’s criteria stipulated the proficiency is submitted at the 
state level, which is a higher level as compared to the measure agreed upon by the SBAE 
teachers. The final SAE metric agreed upon, one American degree every three years, is similar 
to the criteria of directing and submitting one or more State FFA Degree applicants based on 
the student’s SAE program for state consideration. Unlike the previous metric, this agreed upon 
metric is a higher degree than the indicator used by TDE. The three metrics can be used as a 
starting point for accountability and should be reviewed by TDE for appropriateness.  
 
Furthermore, if assessing student skills is not appropriate or cost efficient in the classroom 
instructional component, the SAE component may be able to be used for this evaluation. 
Teachers, employers, and/or placement coordinators may be able to verify students are able to 
perform desired skills in a more cost-effective manner. Future research should explore the use 
of teachers, employers, and/or placement coordinators in conducting skill assessments. 
 
 
 
 



Street et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v2i2.91   95 
 

References 
 
Buchanan, R. (2015) Teacher identity and agency in an era of accountability. Teachers and 

Teaching, 21(6),700–719. https://doi.org/10.1080/13540602.2015.1044329  
 
Conner, N. W., Gates, H., & Stripling, C. T. (2017). Identifying international agricultural concepts 

for secondary agricultural education curriculum. Journal of Agricultural Education, 58(1), 
118–130. https://doi.org/10.5032/jae.2017.01118  

 
Firestone, W. A. (2014). Teacher evaluation policy and conflicting theories of motivation. 

Educational Researcher, 43(2), 100-107. https://doi.org/10.3102/0013189X14521864  
 
Glover, T. A., Reddy, L. A., Kettler, R. J., Kurz, A., & Lekwa, A. J. (2016). Improving high-stakes 

decisions via formative assessment, professional development, and comprehensive 
educator evaluation: The school system improvement project. Teachers College Record, 
118(14), 1–26. https://doi.org/10.1177/1534508418811589 

 
Johnson, T. D., & Mitchell, A. (2014). Guidance for Carl D. Perkins grant recipient [Policy 

Manual]. Division of Post-Secondary and Career Education. 
https://jnc.dc.gov/sites/default/files/dc/sites/osse/publication/attachments/Guidance%
20MASTER%20POLICY%20MANUAL%20%20FINAL%20Rev%209.16.13.pdf  

 
Lavigne, A. L. (2014). Exploring the intended and unintended consequences of high-stakes 

teacher evaluation on schools, teachers, and students. Teachers College Record, 116(1), 
1–29. 
https://www.researchgate.net/publication/284184049_Exploring_the_Intended_and_U
nintended_Consequences_of_High-
Stakes_Teacher_Evaluation_on_Schools_Teachers_and_Students 

 
Lundry, J., Ramsey, J. W., Edwards, M. C., & Robinson, J. S. (2015). Benefits of career 

development events as perceived by school-based, agricultural education teachers. 
Journal of Agricultural Education, 56(1), 43–57. https://doi.org/10.5032/jae.2015.01043 

 
National FFA Organization. (2019). Agricultural Education–About FFA. The National FFA 

Organization: https://www.ffa.org/agricultural-education/ 
 
Phipps, L. J., Osborne, E. W., Dyer, J. E., & Ball, A. (2008). Handbook on Agricultural Education in 

Public Schools (6th ed.). Cengage Learning. 
 
Reform Support Network. (2013). Measuring teacher matters: What different ways of looking at 

student results tell us about teacher effectiveness. 
https://www2.ed.gov/about/inits/ed/implementation-support-unit/tech-
assist/measuring-teaching-matters.pdf 

 



Street et al.  Advancements in Agricultural Development 
 

https://doi.org/10.37433/aad.v2i2.91   96 
 

Shinn, G. C., Briers, G., & Baker, M. (2008). Forecasting doctoral-level content in agricultural 
education: Viewpoints of engaged scholars in the United States. Journal of Agricultural 
Education, 49(1), 121–131. https://doi.org/10.5032/jae.2008.01121 

 
Terry, R. D., & Osborne, E. (2015). Fundamental dimensions and essential elements of 

exemplary local extension units. Journal of Agricultural Education, 56(2), 43-63. 
https://doi.org/10.5032/jae.2015.02043  

 
Touchstone, A. J. L. (2015). Professional development needs of beginning agricultural education 

teachers in Idaho. Journal of Agricultural Education, 56(2), 170–187. 
https://doi.org/0.5032/jae.2015.02170  

 
Warner, L. A., Stubbs, E. Murphrey, T. P., & Huynh, P. (2016). Identification of the competencies 

needed to apply social marketing to extension programing: Results of a Delphi study. 
Journal of Agricultural Education, 57(2), 14–32. https://doi.org/10.5032/jae.2016.02014  

 
 
 
 
 
 
 
© 2021 by authors. This article is an open access article distributed under the terms and conditions of 
the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/). 
 


