Microsoft Word - 10-Agra_42435 1728 Bioscience Journal Original Article Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 DEVELOPMENT AND VALIDATION OF A STANDARD AREA DIAGRAM SET AS ASSESSMENT AID FOR ESTIMATING THE SEVERITY OF BACTERIAL SPOT ON TRI-LOBED LEAVES OF YELLOW PASSION FRUIT DESENVOLVIMENTO E VALIDAÇÃO DE ESCALA DIAGRAMÁTICA COMO AUXÍLIO NA AVALIAÇÃO DA SEVERIDADE DA BACTERIOSE EM FOLHAS TRILOBADAS DO MARACUJAZEIRO AZEDO Anne Pinheiro COSTA1; José Ricardo PEIXOTO1; Luiz Eduardo Bassay BLUM2; Michelle Souza VILELA1; Wagner VENDRAME3 1. Faculdade de Agronomia e Medicina Veterinária, Universidade de Brasília, Brasília, DF, Brasil. annecosta@gmail.com; 2. Departamento de Fitopatologia, Instituto de Ciências Biológicas, Universidade de Brasília, Brasília, DF, Brasil.; 3. Tropical Research and Education Center, University of Florida, Homestead, FL, USA. ABSTRACT: Bacterial spot (Xanthomonas axonopodis pv. passiflorae) significantly reduces yellow passion fruit (Passiflora edulis Sims) yield and longevity. A standard area diagram set (SADs) for severity assessment of bacterial spot on tri-lobed leaves of yellow passion was developed and validated in this study. The SADs consisted of eight severity levels (2; 4; 9; 18; 35; 58; 80; and 94%). For its validation, 20 raters, who initially estimated the disease severity without the aid of the SADs, were divided into four groups (G1 and G3, inexperienced; G2 and G4, experienced). Subsequently, G1 and G2 performed the second evaluation without the SADs, and G3 and G4 completed the second evaluation with the proposed SADs. The accuracy and precision of the assessments were determined by simple linear regression and by the Lin’s concordance correlation coefficient (LCCC). The proposed SADs allowed accurate and precise quantification of bacterial spot severity, increasing the agreement between estimated and actual values. Inexperienced raters benefited the most from the use of the SADs. The increase in accuracy and precision in the non-aided groups, when present, was less pronounced than those increments observed in the SADs-aided groups. The LCCC confirmed the increases in accuracy and precision detected by the linear regression analysis. KEYWORDS: Passiflora edulis Sims. Xanthomonas axonopodis pv. Passiflorae. Phytopathometry INTRODUCTION Bacterial spot (Xanthomonas axonopodis pv. passiflorae) is a disease of difficult control and, as a consequence, is responsible for significant production losses in yellow passion fruit (Passiflora edulis Sims) (ISHIDA et al., 2017). The disease causes intense defoliation and terminal branch drought. These symptoms drastically reduce fruiting, cause plant death, and may ultimately result in total orchard loss (CARVALHO et al., 2015). Bacterial spot is commonly evaluated by visual estimates of disease severity and descriptive keys (VIANA et al., 2014; CASTRO, 2015; NOGUEIRA, 2016). Such keys are prone to various errors due to the inherent subjectivity of the method, which does not allow visual acuity adjustment in the evaluation of the severity levels (CAMPBELL; MADDEN, 1990), compromising accuracy and precision of the estimates. However, subjectivity and error in estimates can be minimized by standard area diagram sets (SADs). SADs consist of a set of illustrations of a diseased plant or plant organs with symptoms at different intensities (ALVES et al., 2015). They are used in plant pathology to increase accuracy (estimates close actual value) and reliability (variation of the estimates among raters) of disease severity quantification (DEL PONTE et al., 2017). SADs-aided severity estimates could subsidize different studies, such as those of disease development under different management conditions (WAMSER et al., 2017), disease impact on plant physiology and yield (RIOS et al., 2017), and mapping of genes for resistance (BASTIANEL et al., 2009). Considering the economic, social, and alimentary importance of yellow passion fruit, the impact of bacterial spot on Brazilian passion fruit production, and the lack of standardized methods for disease severity quantification on tri-lobed leaves of the crop, this study aimed at: (1) developing and validating a SADs to evaluate the bacterial spot severity on tri-lobed leaves of yellow passion fruit; (2) comparing accuracy, precision, and agreement of Received: 16/05/18 Accepted: 10/12/18 1729 Development and validation... COSTA, A. P. et al. Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 non-aided and SADs-aided disease severity estimates; (3) comparing accuracy, precision, and agreement of estimates from inexperienced and experienced raters. MATERIAL AND METHODS Development of the SADs Fifty tri-lobed leaves of yellow passion fruit (BRS Gigante Amarelo, Yellow Master FB200, and genotypes in breeding process) showing symptoms of bacterial spot were collected at Paraná Farm commercial orchard, located in Nucleo Rural Pipiripau, Planaltina, DF, Brazil (47°29’56,92’’ S and 15°30’15,08’’ W, 955 m). The adaxial surface of each leaf was photographed with a digital camera (Canon Powershot SX40 HS, 12.1 megapixels; Canon Inc., Tokyo, Japan), set at the height of 45 cm from the leaf level. The resulting images were analyzed for the diseased area (necrotic + chlorotic) using the image analysis software IMAGE J (SCHNEIDER et al., 2012). The percent leaf area affected by the bacterial spot disease was calculated by the diseased area in relation to the total surface area of the leaf. The area measured using image analysis was considered the actual (true) disease severity and used as a reference for evaluating the accuracy and precision of rater estimates with and without the use of the SADs. The SADs’ upper and lower limits were based on the image analysis-measured minimum and maximum bacterial spot severity in the 50-leaf sample. Intermediate levels were established following logarithmic increments (NUTTER; SCHULTZ, 1995). A standard yellow passion fruit tri-lobed leaf was used as the template, and diagrams with different severity levels were created using the IMAGE J software. The patterns of lesion distribution observed on the actual leaves were maintained. Validation of the SADs The SADs validation was performed with images of 50 leaves with different intensities of symptoms. Twenty raters (10 with previous experience and 10 without previous experience in disease quantification) were selected and divided into four groups of five raters per group (G1 and G3, inexperienced; G2 and G4, experienced). Initially, each group estimated the severity of the disease as percentage, for each of the images from the 50 leaves randomly organized, without the aid of the SADs (non-aided evaluation). Subsequently, the same images were presented to G1 and G2, who performed another non-aided evaluation, and to G3 and G4, who conducted the evaluation using the proposed SADs (SADs-aided evaluation). Data analysis The accuracy, precision, absolute errors, agreement, and inter-reliability of estimates were compared without and with the use of the SADs for both inexperienced and experienced raters. The accuracy and precision of the raters were determined by linear regression between the actual severity (independent variable) and the visually estimated severity (dependent variable). The accuracy of estimates of each rater was determined by a t-test applied to the intercept of linear regression (a) to verify the hypothesis Ho: a = 0, and to the slope of the line (b) to test the hypothesis Ho: b = 1 (P ≤ 0.05). Intercept values significantly different from 0 indicate overestimation (> 0) or underestimation (< 0) of actual severity at low severity levels (constant error). Values of slope of the line that deviate significantly from 1 indicate systematic overestimation (> 1) or underestimation (< 1) of actual severity in all disease intensities (systematic error) (NUTTER; SCHULTZ, 1995). Consequently, the most accurate raters were those whose estimates provided linear regression equations with values of "a" and "b" not significantly different from 0 and 1 by the t-test. The precision of estimates of each rater was obtained by the coefficient of determination of regression analysis (R2) and the variance of absolute errors (the difference between estimated and actual severities) (KRANZ, 1988). Raters with greater precision were those with higher values of R2. Absolute errors were compared by the t-test (P ≤ 0.05). Evaluations of the absolute errors considered the criteria used in disease quantification training programs, such as Distrain (TOMERLIN; HOWELL, 1988) and Disease Pro (NUTTER; WORAWITLIKIT, 1989), which classify raters as excellent (errors up to 5%) or good (errors up to 10%). The mean maximum error (absolute value) was also recorded for each group, indicating the difference of the group’s farthest estimate to the actual severity value. The reproducibility or inter-rater reliability was measured using the R2 values for each pair of raters, based on estimates of non-aided evaluations and SADs-aided evaluations (NUTTER; SCHULTZ, 1995). Accuracy and precision (agreement) of the estimates of each rater, with and without the use of the SADs, was also determined based on the Lin's 1730 Development and validation... COSTA, A. P. et al. Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 concordance correlation coefficient (LCCC; ρc). The LCCC combines measures of accuracy and precision to assess the fit of pairs of observations to the line of concordance (with intercept = 0 and slope = 1), or line 1:1. LCCC is defined by ρc = Cb . r, where Cb, a coefficient calculated based on systematic and constant errors, measures how far the best-fitting line deviates from 45° and is thus a measure of accuracy; and r is the correlation coefficient between estimated severity (Y) and actual severity (X), which measures precision (variation) or the scattering of points around the best-fitting line. When there is perfect agreement between estimated and actual severity, the points fall on the concordance line. As a result, r = 1, Cb = 1, and ρc = 1 (LIN, 1989; BOCK et al., 2010). Linear regressions and absolute error analyses were performed using the Genes software (v. 1990.2017.37). The LCCC was calculated using MedCalc software (v. 17.9.7). RESULTS AND DISCUSSION The actual severity values of bacterial spot on tri-lobed leaves of yellow passion fruit ranged from 2 to 93.7%. The lesions exhibited typical symptomatic patterns of the disease, with initial small spots, usually initiated at the margins, which grow and coalesce and could ultimately cover the entire leaf area (JUNQUEIRA et al., 2016). These severity values are often observed in the field and under protected cultivation since bacterial spot is difficult to control, and no cultivar with satisfactory resistance level has been reported to date (ISHIDA & HALFELD-VIEIRA, 2009). Based on the severity levels observed, a black and white 8- diagram SADs (2; 4; 9; 18; 35; 58; 80; and 94%) was proposed (Figure 1). The number of diagrams used to represent the extent of the disease severity was similar to the number used in SADs proposed for other pathosystems involving different Xanthomonas species (BELASQUE JR. et al., 2005; NASCIMENTO et al., 2005; LIMA et al., 2013). Figure 1. Standard area diagram set (SADs) for bacterial spot severity assessment on tri-lobed leaves of yellow passion fruit (Passiflora edulis Sims). The numbers in the diagrams represent the leaf area with necrotic symptoms of the disease (%). Brasilia, DF, Brazil, 2018. Accuracy is defined as the exactness of a systematic error-free measure and is measured by the intercept and slope coefficient of a linear regression between actual severity and estimated severity (BERGAMIN FILHO; AMORIM, 1996). Accurate assessments have intercept values equal to 1731 Development and validation... COSTA, A. P. et al. Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 0 and slope coefficient values equal to 1 (NUTTER; SCHULTZ, 1995). Considerable variation in the constant and systematic errors was verified in the first assessment among raters of all groups, with underestimated and overestimated severity values observed. The utilization of the SADs reduced both types of error in the second evaluation of G3. The estimated values tended to approach the reference values for the absence of constant (a = 0) and systematic errors (b = 1), in addition to reducing the amplitude of the underestimated and overestimated values in G3 and G4 (Table 1). Consequently, SADs-aided estimates were closer to the actual severity value (Figures 2-10). Table 1. Intercepts (a), slope coefficients (b), and coefficients of determination (R2) of linear regression for actual severity versus estimated severity of bacterial spot (Xanthomonas axonopodis pv. passiflorae) on tri-lobed leaves of yellow passion fruit (Passiflora edulis Sims). Brasilia, DF, Brazil, 2018. Group Rater Evaluation 1 Evaluation 2 1 Inexperienced No SADs No SADs a b R2 a b R2 1 -1.21 1.12* 0.91 2.79* 0.93* 0.95 2 -3.38* 1.17* 0.98 4.07* 0.97 0.96 3 -1.08 1.00 0.97 -2.45* 1.06* 0.98 4 5.00* 0.90* 0.97 6.58* 0.92* 0.98 5 -3.20* 1.06* 0.97 2.30* 0.98 0.98 Mean -0.77 1.05 0.96 2.66 0.97 0.97 2 Experienced No SADs No SADs 6 -1.99* 0.99 0.98 -1.52* 0.97 0.99 7 -0.65 0.98 0.97 7.71* 0.91* 0.98 8 2.78* 0.94* 0.97 1.84* 0.95* 0.98 9 3.21* 0.87* 0.95 3.53* 0.92* 0.96 10 -0.60 0.96 0.98 2.94* 0.94* 0.99 Mean 0.55 0.95 0.97 2.90 0.94 0.98 3 Inexperienced No SADs With SADs 11 3.47* 0.87* 0.97 1.93* 1.02 0.98 12 7.37* 0.84* 0.95 4.19* 0.96 0.98 13 3.04* 1.02 0.96 2.25* 1.01 0.96 14 5.33* 0.91 0.86 1.50* 1.04 0.97 15 7.73* 0.97 0.94 0.51 1.04 0.98 Mean 6.66 0.91 0.93 2.10 1.02 0.98 4 Experienced No SADs With SADs 16 -1.54 1.05 0.97 4.50* 0.99 0.95 17 -2.89* 1.08* 0.98 -0.06 1.08* 0.98 18 1.02 1.06* 0.97 2.65* 1.01 0.97 19 10.70* 0.87* 0.95 2.28* 1.00 0.98 20 -3.24* 1.06* 0.97 1.54 1.02 0.97 Mean 0.81 1.02 0.97 2.18 1.02 0.97 * indicate that the null hypothesis (a = 0 or b = 1) was rejected by t-test (P ≤ 0.05). The greatest increases in accuracy with the SADs were verified for the group of inexperienced raters. This improvement was notably influenced by the reduction of systematic errors in G3 and G4 since 100% (G3) and 80% (G4) of the raters presented systematic error-free estimates while 20% (G3) and 40% (G4) exhibited estimates free of constant errors. The increase in accuracy was also influenced by the reduction of the mean intercept value in G3, which decreased from 6.66 in the first evaluation to 2.10 in the second evaluation, demonstrating attenuation in the overestimated values initially observed. Conversely, increases in constant and/or systematic errors were identified in the second non-aided evaluation in G1 and G2 (Table 1). Disease severity assessments may vary considerably among individuals, since factors affecting accuracy and precision of estimates may depend on the rater (innate ability, experience, and 1732 Development and validation... COSTA, A. P. et al. Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 training) and plant lesions or organs characteristics (BOCK et al., 2016). For instance, small lesions occurring in a higher number usually lead to disease overestimation at low levels of disease severity (< 10%) (BOCK et al., 2010; SCHWANCK; DEL PONTE, 2014), whereas large and fewer lesions tend to be underestimated (SPOLTI et al., 2011; GONZÁLEZ-DOMÍNGUEZ et al., 2014). The precision of the severity estimates was high in all groups, with or without the SADs aid (Table 1). Without using the SADs, G3 raters showed R2 values between 0.86 and 0.97 (mean 0.93), and G4 raters exhibited R2 values between 0.95 e 0.98 (0.97). When the SADs was used, R2 values varied from 0.96 to 0.98 (0.98) in G3 and from 0.95 to 0.98 (0.97) in G4. As estimates were highly precise in all groups in the first assessment, little increase in R2 values was identified in the second evaluation, except for G3, with a 5.4% increment (Table 1). The absolute errors were significantly reduced in the inexperienced groups (Figures 2 to 9) in the second evaluation (Table 2), resulting in lower data dispersion in the regression analysis (Figures 3 and 7) and lower error distribution amplitude (Figure 10). Raters with previous experience in disease quantification have internally and, often, empirically developed their visual calibration process. Since experience and training reduce the subjectivity and errors of the estimates, experienced raters tend to be more precise than those with no previous experience, even if the experienced rater ability was subjectively developed (NUTTER; SCHULTZ, 1995; BOCK et al., 2010). As a consequence, inexperienced raters usually have superior increments on accuracy and precision when SADs are used as compared to the experienced raters (YADAV et al., 2013; VENTURINI et al., 2015; NUÑEZ et al., 2017). Table 2. Absolute errors (estimated severity – actual severity) from bacterial spot (Xanthomonas axonopodis pv. passiflorae) severity estimates on tri-lobed leaves of yellow passion fruit (Passiflora edulis Sims). Brasilia, DF, Brazil, 2018. Group Rater Evaluation 1 Evaluation 2 1 Inexperienced No SADs No SADs 1 5.6 a 3.7 a 2 4.3 a 4.3 a 3 2.7 a 2.9 a 4 4.5 a 5.4 a 5 4.0 b 2.9 a Mean 4.2 a 3.9 a 2 Experienced No SADs No SADs 6 3.0 a 2.4 a 7 2.7 a 6.3 b 8 3.6 a 2.9 a 9 4.3 a 4.1 a 10 2.4 a 2.6 a Mean 3.2 a 3.6 a 3 Inexperienced No SADs With SADs 11 3.8 a 3.0 a 12 5.9 b 3.9 a 13 4.2 a 3.5 a 14 5.5 b 3.3 a 15 7.4 b 2.7 a Mean 5.4 b 3.3 a 4 Experienced No SADs With SADs 16 3.4 a 5.3 b 17 3.2 a 2.8 a 18 3.7 a 3.6 a 19 8.7 b 3.3 a 20 3.4 a 3.3 a Mean 4.5 a 3.7 a Different letters in the same row indicate significant differences (Student´s t-test, P ≤ 0.05). 1733 Development and validation... COSTA, A. P. et al. Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 Figure 2. Bacterial spot (Xanthomonas axonopodis pv. passiflorae) severity estimation on tri-lobed leaves of yellow passion fruit (Passiflora edulis Sims) by group 1 (inexperienced raters), without the standard area diagram set (SADs) in the 1st evaluation. Solid line = linear regression of actual severity x estimated severity. Dotted line = perfect agreement (linear regression of actual severity = estimated severity). Brasilia, DF, Brazil, 2018. 1734 Development and validation... COSTA, A. P. et al. Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 Figure 3. Bacterial spot (Xanthomonas axonopodis pv. passiflorae) severity estimation on tri-lobed leaves of yellow passion fruit (Passiflora edulis Sims) by group 1 (inexperienced raters), without the standard area diagram set (SADs) in the 2nd evaluation. Solid line = linear regression of actual severity x estimated severity. Dotted line = perfect agreement (linear regression of actual severity = estimated severity). Brasilia, DF, Brazil, 2018. 1735 Development and validation... COSTA, A. P. et al. Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 Figure 4. Bacterial spot (Xanthomonas axonopodis pv. passiflorae) severity estimation on tri-lobed leaves of yellow passion fruit (Passiflora edulis Sims) by group 2 (experienced raters), without the standard area diagram set (SADs) in the 1st evaluation. Solid line = linear regression of actual severity x estimated severity. Dotted line = perfect agreement (linear regression of actual severity = estimated severity). Brasilia, DF, Brazil, 2018. 1736 Development and validation... COSTA, A. P. et al. Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 Figure 5. Bacterial spot (Xanthomonas axonopodis pv. passiflorae) severity estimation on tri-lobed leaves of yellow passion fruit (Passiflora edulis Sims) by group 2 (experienced raters), without the standard area diagram set (SADs) in the 2nd evaluation. Solid line = linear regression of actual severity x estimated severity. Dotted line = perfect agreement (linear regression of actual severity = estimated severity). Brasilia, DF, Brazil, 2018 1737 Development and validation... COSTA, A. P. et al. Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 Figure 6. Bacterial spot (Xanthomonas axonopodis pv. passiflorae) severity estimation on tri-lobed leaves of yellow passion fruit (Passiflora edulis Sims) by group 3 (inexperienced raters), without the standard area diagram set (SADs) in the 1st evaluation. Solid line = linear regression of actual severity x estimated severity. Dotted line = perfect agreement (linear regression of actual severity = estimated severity). Brasilia, DF, Brazil, 2018. 1738 Development and validation... COSTA, A. P. et al. Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 Figure 7. Bacterial spot (Xanthomonas axonopodis pv. passiflorae) severity estimation on tri-lobed leaves of yellow passion fruit (Passiflora edulis Sims) by group 3 (inexperienced raters), with the standard area diagram set (SADs) in the 2nd evaluation. Solid line = linear regression of actual severity x estimated severity. Dotted line = perfect agreement (linear regression of actual severity = estimated severity). Brasilia, DF, Brazil, 2018. 1739 Development and validation... COSTA, A. P. et al. Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 Figure 8. Bacterial spot (Xanthomonas axonopodis pv. passiflorae) severity estimation on tri-lobed leaves of yellow passion fruit (Passiflora edulis Sims) by group 4 (experienced raters), without the standard area diagram set (SADs) in the 1st evaluation. Solid line = linear regression of actual severity x estimated severity. Dotted line = perfect agreement (linear regression of actual severity = estimated severity). Brasilia, DF, Brazil, 2018. 1740 Development and validation... COSTA, A. P. et al. Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 Figure 9. Bacterial spot (Xanthomonas axonopodis pv. passiflorae) severity estimation on tri-lobed leaves of yellow passion fruit (Passiflora edulis Sims) by group 4 (experienced raters), with the standard area diagram set (SADs) in the 2nd evaluation. Solid line = linear regression of actual severity x estimated severity. Dotted line = perfect agreement (linear regression of actual severity = estimated severity). Brasilia, DF, Brazil, 2018. 1741 Development and validation... COSTA, A. P. et al. Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 Figure 10. Distribution of absolute errors (estimated severity – actual severity) of the bacterial spot (Xanthomonas axonopodis pv. passiflorae) estimates on tri-lobed leaves of yellow passion fruit (Passiflora edulis Sims) in the 1st evaluation, without the standard area diagram set (SADs), in groups 1 (A), 2 (B), 3 (C), and 4 (D); and in the 2nd evaluation, without the aid of the SADs, in groups 1 (E) and 2 (F), and with the SADs, in groups 3 (G) and 4 (H). Brasilia, DF, Brazil, 2018. G1 and G2 presented reductions of 9.6% and 5.3%, respectively, in the mean maximum error in the second non-aided evaluation. Conversely, reductions of 36.3% (G3) and 8.5% (G4) were observed for the SADs-aided groups (Table 3, Figure 10). Therefore, errors varied from -40.0 to +23.0 in G3 (Figure 10C) and from -18.0 to +17.8 in G4 (Figure 10D) in the non-aided evaluations. When the SADs was used, a marked reduction was verified in G3 (-8.0 to +21.0) (Figure 10E) while in G4, errors ranged from -12.0 to +16.4 (Figure 10F). Errors of greater magnitudes were identified in the second evaluation of G2, increasing the percentage of estimates with errors above 10% (-10 to +10). In contrast, an increase in the number of estimates with errors up to 10% was recorded when the SADs was used. These findings represent an increase of 10.5 and 5.9% as compared to the values observed in the first disease assessment in G3 and G4, respectively (Table 3). Although a similar behavior was verified in G1, the increase in the number of estimates with errors up to 10% was less pronounced than those observed in the SADs-aided groups. The percentage of estimates with errors below 5% (-5 to +5) also increased by 34% (G3) and 14.3% (G4) as compared to the non-aided evaluation. Although G1 had a higher number of estimates with errors up to 5% in the second non- aided evaluation, the recorded 2.8% increase was lower than the increments detected in G3 and G4 (Table 3). The gains in inter-rater reliability, or reproducibility of the estimates, are critical to guarantee that multiple raters are more uniform when estimating disease severity in the same experiment (CORREIA et al., 2017). 1742 Development and validation... COSTA, A. P. et al. Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 Reproducibility can be quantified by the R2 of the regression line generated by the estimates of two raters. Values close to 1 indicate similar estimates (BELASQUE et al., 2005). The reproducibility of the estimates between raters was greater with the use of the SADs. Non-aided evaluations had R2 values lower than 0.90 in 60% (G3) and 10% (G4) of the comparisons between raters. When SADs were used, R2 values were higher than 0.90 for 100% of the estimates in both groups, with mean R2 values of 0.96 (G3) and 0.95 (G4). These results indicate high reliability of the estimates. Table 3. Mean maximum error in absolute value (MEAV), 10% (x ± 10) and 5% (x ± 5) error range of the severity estimates as related to the actual severity of bacterial spot (Xanthomonas axonopodis pv. passiflorae) on tri-lobed leaves of yellow passion fruit (Passiflora edulis Sims). Brasília, DF, Brazil, 2018. Evaluation Parameter Group 1 2 3 4 1 MEAV 15.6 13.2 22.3 15.3 % x ± 5 72.0 78.4 58.8 67.2 % x ± 10 92.4 97.6 88.0 87.6 2 MEAV 14.1 12.5 14.2 14.0 % x ± 5 74.0 74.4 78.8 76.8 % x ± 10 96.4 97.2 97.2 92.8 Our findings demonstrate that SADs-aided severity estimates were more accurate and precise for most raters as compared to non-aided estimates (Tables 1 and 4). The SADs also provided high reproducibility, especially for inexperienced raters. As a result, the proposed SADs was useful in reducing the variability of the estimates and amplitude of the variations, especially in the group of inexperienced raters. These findings corroborate other studies which investigated the effect of the evaluator's experience on both accuracy and precision, concluding that inexperienced raters tend to respond more efficiently to the use of SADs as compared to experienced raters (YADAV et al. 2013; GONZÁLEZ-DOMÍNGUEZ et al., 2014; SOUSA et al., 2014). The LCCC provides an overall measure of agreement/concordance between estimated and actual values and has been used in plant disease assessments (GONZÁLEZ-DOMÍNGUEZ et al., 2014; NICOLI et al., 2015; CORREIA et al., 2017). The LCCC analysis detected increments in accuracy and precision higher than those identified by linear regression analysis (Table 4). Non-aided estimates were highly accurate and precise in all groups, resulting in high agreement values. Therefore, only a slight increase in precision in G3 and accuracy in G3 and G4 were verified when the SADs was used. Little increase in precision (G1 and G2) and reduced accuracy (G2) were detected in the second non- aided evaluation (Table 4). The use of SADs resulted in increased agreement (ρc) for 100% of the raters in G3 and 80% in G4, as also verified by linear regression analysis. When SADs was not used, ρc varied from 0.92 to 0.98 (G3) and from 0.90 to 0.98 (G4). However, a variation from 0.98 to 0.99 (G3) and from 0.95 to 0.98 (G4) was recorded when the SADs was used, corresponding to increments of 3.2 and 2.1% in G3 and G4 agreement, respectively. These results indicate that SADs-aided estimates were closer to the concordance line when compared to the previous non-aided estimates (Table 4, Figures 6-9). No increase was verified in G1 or G2 mean agreement values. Table 4. Correlation coefficient between estimated severity and actual severity (r), bias correction factor (Cb), and Lin’s concordance correlation coefficient (ρc) for bacterial spot (Xanthomonas axonopodis pv. passiflorae) severity estimates on tri-lobed leaves of yellow passion fruit (Passiflora edulis Sims). Brasilia, DF, Brazil, 2018. Group Rater Evaluation 1 Evaluation 2 1 Inexperienced No SADs No SADs r Cb ρc r Cb ρc 1743 Development and validation... COSTA, A. P. et al. Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 1 0.96 0.99 0.94 0.98 1.00 0.97 2 0.99 0.99 0.98 0.98 0.99 0.97 3 0.99 1.00 0.99 0.99 1.00 0.99 4 0.98 0.99 0.97 0.99 0.97 0.96 5 0.98 0.99 0.98 0.99 1.00 0.99 Mean 0.98 0.99 0.97 0.99 0.99 0.97 2 Experienced No SADs No SADs 6 0.99 1.00 0.98 0.99 1.00 0.99 7 0.99 1.00 0.98 0.99 0.96 0.95 8 0.98 1.00 0.98 0.99 1.00 0.99 9 0.97 0.99 0.97 0.98 0.99 0.97 10 0.99 1.00 0.99 0.99 1.00 0.99 Mean 0.98 1.00 0.98 0.99 0.99 0.98 3 Inexperienced No SADs With SADs 11 0.98 0.99 0.98 0.99 0.99 0.99 12 0.97 0.97 0.94 0.99 0.99 0.98 13 0.98 0.99 0.97 0.98 0.99 0.98 14 0.93 0.99 0.92 0.99 0.99 0.98 15 0.97 0.95 0.92 0.99 1.00 0.99 Mean 0.97 0.98 0.95 0.99 0.99 0.98 4 Experienced No SADs With SADs 16 0.98 1.00 0.98 0.97 0.98 0.95 17 0.99 0.99 0.98 0.99 0.99 0.98 18 0.98 0.99 0.97 0.99 0.99 0.98 19 0.97 0.93 0.90 0.99 0.99 0.98 20 0.99 0.99 0.98 0.98 1.00 0.98 Mean 0.98 0.98 0.96 0.98 0.99 0.98 The SADs developed and validated in this study will be useful for bacterial spot severity quantification on tri-lobed leaves of yellow passion fruit. The diagrams here proposed will be especially important in studies involving multiple raters since its use resulted in more accurate and precise estimates from experienced and, particularly, inexperienced raters. Consequently, the SADs may be used to aid genotype selection in breeding programs, during screening for disease resistance in both seedlings and adult plants, under field and protected cultivation conditions. RESUMO: A bacteriose (Xanthomonas axonopodis pv. passiflorae) reduz significativamente a produção e longevidade do maracujazeiro azedo (Passiflora edulis Sims). Uma escala diagramática para a avaliação da severidade da bacteriose em folhas trilobadas do maracujazeiro azedo foi desenvolvida e validada neste estudo. A escala diagramática apresentou oito níveis de severidade (2; 4; 9; 18; 35; 58; 80 e 94%). Para a sua validação, os 20 avaliadores foram divididos em quatro grupos (G1 e G3, sem experiência; G2 e G4, com experiência), que inicialmente estimaram a severidade da doença sem auxílio da escala. Posteriormente, G1 e G2 fizeram outra avaliação sem escala, e G3 e G4 realizaram a avaliação com a escala proposta. A acurácia e a precisão das estimativas foram determinadas por regressão linear simples e pelo coeficiente de correlação de concordância de Lin (LCCC). A escala diagramática proposta permitiu quantificar a severidade da bacteriose de forma acurada e precisa, aumentando a concordância entre os valores estimados e os reais. Os avaliadores inexperientes foram os mais beneficiados pelo uso da escala. O aumento da acurácia e precisão nos grupos que realizaram dupla avaliação sem escala, quando ocorreu, foi mais discreto que os incrementos observados nos grupos que utilizaram a escala. O LCCC confirmou os incrementos da acurácia e precisão detectados pela análise de regressão linear. PALAVRAS-CHAVE: Passiflora edulis Sims. Xanthomonas axonopodis pv. Passiflorae. Fitopatometria 1744 Development and validation... COSTA, A. P. et al. Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 REFERENCES ALVES, G. C. S.; CASTRO SANTOS, L.; DUARTE, H. S. S.; DIAS, V.; ZAMBOLIM, L.; ROCHA, M. R. Escala diagramática para quantificação da ferrugem da folha do trigo. Multi-Science Journal, v. 1, n. 1, p.128- 133, 2015. https://doi.org/10.33837/msj.v1i1.59 BASTIANEL, M.; CRISTOFANI-YALY, M.; OLIVEIRA, A.C.; FREITAS-ASTU, J.; GARCIA, A. A. F.; RESENDE, M. D. V.; RODRIGUES, V.; MACHADO, M. A. Quantitative trait loci analysis of citrus leprosis resistance in an interspecific backcross family of (Citrus reticulata Blanco × C. sinensis L. Osbeck) × C. sinensis L. Osb. Euphytica, v. 169, n. 1, p. 101-111, 2009. https://doi.org/10.1007/s10681-009-9950-3 BELASQUE JÚNIOR, J.; BASSANEZI. R. B., SPÓSITO, M. B.; RIBEIRO, L. M.; JESUS JÚNIOR, W. C.; AMORIM, L. Escalas diagramáticas para avaliação da severidade do cancro cítrico. Fitopatologia Brasileira, v. 30, n. 4, p. 387-393, 2005. https://doi.org/10.1590/S0100-41582005000400008 BERGAMIN FILHO, A.; AMORIM, L. Doenças de plantas tropicais: epidemiologia e controle econômico. São Paulo: Agronômica Ceres, 1996. 299p. BOCK, C. H.; POOLE, G.; PARKER, P. E.; GOTTWALD, T. R. Plant disease severity estimated visually, by digital photography and image analysis, and by hyperspectral imaging. Critical Reviews in Plant Sciences, v. 29, p. 59-107, 2010. https://doi.org/10.1080/07352681003617285 BOCK, C. H.; CHIANG, K. S.; DEL PONTE, E. M. Accuracy of plant specimen disease severity estimates: concepts, history, methods, ramifications and challenges for the future. CAB Reviews: Perspectives in Agriculture, Veterinary Science, Nutrition and Natural Resources, v. 11, n. 39, p. 1-13, 2016. https://doi.org/10.1079/PAVSNNR201611032 CAMPBELL, C. L.; MADDEN, L. V. Introduction to plant disease epidemiology. 1. ed. New York: John Wiley, 1990. 532p. CARVALHO, S. L. C.; STENZEL, N. M. C.; AULER, P. A. M. Maracujá-amarelo: Recomendações técnicas para cultivo no Paraná. Londrina: IAPAR, 2015. 54 p. CASTRO, A. P. G. Desempenho agronômico, diversidade genética e avaliação de doenças em progênies de maracujazeiro-azedo. 2015. 204 f. Tese (Doutorado em Agronomia) – Curso de Pós Graduação em Agronomia, Universidade de Brasília, Brasília, 2015. CORREIA, K. C.; QUEIROZ, J. V. J.; MARTINS, R. B.; NICOLI, A.; DEL PONTE, E. M.; MICHEREFF, S. J. Development and evaluation of a standard area diagram set for the severity of phomopsis leaf blight on eggplant. European Journal of Plant Pathology, v. 149, p. 269-276, 2017. https://doi.org/10.1007/s10658- 017-1184-y DEL PONTE, E. M.; PETHYBRIDGE, S. J.; BOCK, C. H.; MICHEREFF, S. J.; MACHADO, F. J.; SPOLTI, P. Standard Area Diagrams for Aiding Severity Estimation: Scientometrics, Pathosystems, and Methodological Trends in the Last 25 Years. Phytopathology, v. 107, n. 10, p. 1161-1174, 2017. https://doi.org/10.1094/PHYTO-02-17-0069-FI GONZÁLEZ-DOMÍNGUEZ, E.; MARTINS, R. B.; DEL PONTE, E. M.; MICHEREFF, S. J.; GARCÍA- JIMÉNEZ, J.; ARMENGOL, J. Development and validation of a standard area diagram set to aid assessment of severity of loquat scab on fruit. European Journal of Plant Pathology, v. 139, p. 413-422, 2014. https://doi.org/10.1007/s10658-014-0400-2 ISHIDA, A. K. N.; HALFED-VIEIRA, B. A. Mancha-Bacteriana do Maracujazeiro (Xanthomonas axonopodis pv. passiflorae): Etiologia e Estratégias de Controle. Belém: Embrapa Amazônia Oriental, 2009. 23 p. 1745 Development and validation... COSTA, A. P. et al. Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 ISHIDA, A. K. N.; PROTAZIO, D. C.; OLIVEIRA, L. C. Preservação de Isolados de Xanthomonas axonopodis pv. passiflorae em Água Destilada Esterilizada. Belém: Embrapa Amazônia Oriental, 2017. 18 p. JUNQUEIRA, N. T. V.; SUSSEL, A. A. B.; JUNQUEIRA, K. P.; ZACARONI, A. B.; BRAGA, M. F. Doenças. In: FALEIRO, F. G.; JUNQUEIRA, N. T. V. (Ed.). Maracujá: O produtor pergunta, a Embrapa responde. Brasília, DF: Embrapa Cerrados, 2016. p. 169-180. (Coleção 500 perguntas, 500 respostas). KRANZ, J. Measuring plant disease. In: KRANZ, J.; ROTEM, J. (Ed.). Experimental techniques in plant disease epidemiology. Heidelberg: Springer-Verlag, 1988. p. 35-50. https://doi.org/10.1007/978-3-642-95534- 1_4 LIMA, H. E.; NECHET, K. L.; HALFELD-VIEIRA, B. A.; OLIVEIRA, J. R.; DUARTE, H. S. S.; QUEIROZ, E. S.; OLIVEIRAM F. L. Elaboração e validação de escalas diagramáticas para avaliação da severidade da mancha-bacteriana do feijão-caupi em cultivares com trifólios morfologicamente distintos. Ciência Rural, v. 43, n. 10, p. 1735-1743, 2013. https://doi.org/10.1590/S0103-84782013001000001. LIN, L. I. A concordance correlation coefficient to evaluate reproducibility. Biometrics, v. 45, p. 255-268, 1989. https://doi.org/10.2307/2532051 NASCIMENTO, A. R. P.; MICHEREFF, S. J.; MARIANO, R. L. R.; GOMES, A. M. A. Elaboração e validação de escala diagramática para cancro bacteriano da videira. Summa Phytopathologica, v. 31, n. 1, p. 59-64, 2005. NICOLI, A.; COSTA, R. V.; COTA, L. V.; SILVA, D. D.; ZAMBOLIM, L.; LANZA, F. E.; GUIMARÃES, D. P.; LANDAU, E. C. Validação de escala diagramática para quantificação da severidade da antracnose do colmo do milho. Ciência Rural, v. 45, n. 10, p. 1720-1726, 2015. https://doi.org/10.1590/0103- 8478cr20141510 NOGUEIRA, I. Caracterização agronômica e físico-química de progênies de maracujazeiro azedo (Passiflora edulis Sims) no Distrito Federal. 2016. 113 f. Dissertação (Mestrado em Agronomia) - Curso de Pós-graduação em Agronomia, Universidade de Brasília, Brasília, 2016. NUÑEZ, A. M. P.; MONTEIRO, F. P.; PACHECO, L. P.; RODRÍGUEZ, G. A. A.; NOGUEIRA, C. C. A.; PINTO, F. A. M. F.; MEDEIROS, F. A. V.; SOUZA, J. T. Development and validation of a diagrammatic scale to assess the severity of black rot of crucifers in Kale. Journal of Phytopathology, v. 165, n. 3, p. 195-203, 2017. https://doi.org/10.1111/jph.12550 NUTTER, F. W.; WORAWITLIKIT, O. Disease.Pro: a computer program for evaluating and improving a person ability to assess disease proportion. Phytopathology, v. 79, p. 1135, 1989. https://doi.org/10.1094/Phyto-79-958 NUTTER JR., F. W.; SCHULTZ, P. M. Improving the accuracy and precision of disease assessment: selection of methods and use of computer-aided training programs. Canadian Journal of Plant Pathology, v. 17, n. 2, p. 174-184, 1995. https://doi.org/10.1080/07060669509500709 RIOS, J. A.; RIOS, V. S.; AUCIQUE-PÉREZ, C. E.; CRUZ, M. F. A.; MORAIS, L. E.; DAMATTA, F. M.; RODRIGUES, F. A. Alteration of photosynthetic performance and source–sink relationships in wheat plants infected by Pyricularia oryzae. Plant Pathology, v. 66, n. 9, p. 1496-1507, 2017. https://doi.org/10.1111/ppa.12693 SCHNEIDER, C. A.; RASBAND, W. S.; ELICEIRI, K. W. NIH Image to ImageJ: 25 years of image analysis. Nature Methods, v. 9, p. 671-675, 2012. https://doi.org/10.1038/nmeth.2089 1746 Development and validation... COSTA, A. P. et al. Biosci. J., Uberlândia, v. 35, n. 6, p. 1728-1746, Nov./Dec. 2019 http://dx.doi.org/10.14393/BJ-v35n6a2019-42435 SCHWANCK, A. A.; DEL PONTE, E. M. Accuracy and reliability of severity estimates using linear or logarithmic disease diagram sets in true colour or black and white: A study case for rice brown spot. Journal of Phytopatholy, v. 162, p. 670-682, 2014. https://doi.org/10.1111/jph.12246 SOUSA, S. C. R.; SANTOS, G. R.; RODRIGUES, A. C.; BONIFÁCIO, A.; DALCIN, M. S.; JULIATTI, F. C. Escala diagramática para avaliação da severidade do crestamento gomoso do caule em melancia. Bioscience Journal, v. 30, n. 5, p. 1314-1324, 2014. SPOLTI, P.; SCHNEIDER, L.; SANHUEZA, R. M. V.; BATZER, J. C.; GLEASON, M. L.; DEL PONTE, E. M. Improving sooty blotch and flyspeck severity estimation on apple fruit with the aid of standard area diagrams. European Journal of Plant Pathology, n. 129, p. 21-29, 2011. https://doi.org/10.1007/s10658- 010-9636-7 TOMERLIN, J. R.; HOWELL, T. A. Distrain: a computer program for training people to estimate disease severity on cereal leaves. Plant Disease, v. 72, p. 455-459, 1988. VENTURINI, M. T.; SANTOS, L. R.; OLIVEIRA, E. J. Development of a diagrammatic scale for the evaluation of postharvest physiological deterioration in cassava roots. Pesquisa Agropecuária Brasileira, v. 50, n. 8, p. 658-668, 2015. https://doi.org/10.1590/S0100-204X2015000800004 VIANA, C. A. S.; PIRES, M. C.; PEIXOTO, J. R.; JUNQUEIRA, N. T. V.; BLUM, L. E. B. Genótipos de maracujazeiro-azedo com resistência à bacteriose. Bioscience Journal, v. 30, supplement 2, p. 591-598, 2014. YADAV, N. V.; VOS, S. M.; BOCK, C. H.; WOOD, B. W. Development and validation of standard area diagrams to aid assessment of pecan scab symptoms on fruit. Plant Pathology, v. 62, n. 2, p. 325-335, 2013. https://doi.org/10.1111/j.1365-3059.2012.02641.x WAMSER, A. F.; VALMORBIDA, J.; SUZUKI, A.; HAHN, L.; MUELLER, S.; BECKER, W. F.; ANDERSON LUIZ FELTRIM, A. F.; ENDER, M. M. Planting density and arrangement for the mechanized spraying of vertically staked tomatoes. Horticultura Brasileira, v. 35, n. 4, p. 519-526, 2017. https://doi.org/10.1590/s0102-053620170408