Microsoft Word - 145-Manuscript-1813-1-11-20211011.docx Zagonel et al. Advancements in Agricultural Development Volume 2, Issue 3, 2021 agdevresearch.org 1. Anissa M. Zagonel, Research Coordinator, UF/IFAS Center for Public Issues Education in Agriculture and Natural Resources, University of Florida, 1408 Sabal Palm Drive, 2nd Floor, PO Box 110320, Gainesville, FL 32611 azagonel@ufl.edu, https://orcid.org/0000-0001-6359-5820 2. Lauri M. Baker, Associate Professor, University of Florida, PO Box 112060 Gainesville, FL 32611, lauri.m.baker@ufl.edu, https://orcid.org/0000-0002-4241-6077 3. Joelle Covarrubias, Undergraduate Research Assistant, University of Florida, j.covarrubias@ufl.edu, https://orcid.org/0000-0003-4131-0112 4. Angela B. Lindsey, Assistant Professor, University of Florida, 3014C McCarty D, Gainesville, FL 32611 ablindsey@ufl.edu, https://orcid.org/0000-0002-9859-7962 39 Credibility in crisis: Determining the availability and credibility of online food supply chain resources during the COVID-19 pandemic A. Zagonel1, L. Baker2, J. Covarrubias3, A. Lindsey4 Abstract Disruptions from COVID-19 forced agricultural business owners to navigate the uncertainty of market disruptions with limited information. As an effect, the quality of information available for agricultural businesses to adapt to changes was a concern. The purpose of this exploratory study was to determine the availability and credibility of resources for agricultural businesses to make informed decisions about food markets during COVID-19. Source credibility was the guiding framework to achieve the research objectives of 1) Describe resources available related to impacts of COVID-19 on the food supply chain, 2) Determine the credibility of available resources. A quantitative content and textual analyses were employed. Results revealed 401 terms used to describe resources (n = 779). Eleven of the top 36 terms were used over 100 times. These were: farmer, resources, farm, market, business, local, health, safe, supply, agriculture, and chain. The majority of resources (66%, f = 514) were mid-level credible sources (industry/business organization, online/print news source, nonprofit), and 32.2% (f = 251) were of the highest credibility (university scientists, USDA scientist, Extension). Implications of this work show an opportunity for university and Extension systems to publish resources and serve as credible sources related to this particular crisis. Keywords Source credibility, quantitative content analysis, textual analysis, online resources Zagonel et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v2i3.145 40 Introduction and Problem Statement A crisis is a complex event that causes or threatens to cause widespread disaster among a whole social system, often with an urgent timeline forcing rapid responses (Pauchant & Mitroff, 1990). In early 2020, COVID-19 swept across the U.S. creating subsequent crises with system shattering impacts (Sahin et al., 2020), such as obstructive changes in social behaviors (Nakat & Bou-Mitri, 2021). The impacts to agriculture were great with disruptions in food supply chains (Duvall, 2020) and increased need for alternative methods of selling agricultural products to meet new needs (Nelson, 2020). These social behavior changes, specific to agriculture, have continued well into 2021 and are observed as changes in consumers’ food buying patterns and disruptions to supply chains (Nakat & Bou-Mitri, 2021). With the accessibility of traditional avenues and markets being limited or closed, agricultural business owners were left to navigate through the uncertainty and indirect impacts of COVID- 19 (Lioutas & Charatsari, 2021). During the pandemic, agriculture and natural resource business owners expressed concern over the quality of information available to help inform decision making (Baker, Kandzer, Rampold, et al., 2020). A stimulus policy report recommended that food and farm business entities should be included in small business support programs and receive the same technical assistance and resources (Thilmany et al., 2020). Extension systems have been positioned as a credible source with adequate online program offerings to serve as a mediator and fill these information gaps in times of crises (Narine & Meier, 2020). However, little is known about what resources currently exist for agricultural business owners related specifically to COVID-19 food supply disruptions. Because of this problem, this study sought to explore and identify available resources and assess the quality of resources, in terms of authoring source credibility. With a better understanding of what topic areas are already covered and the credibility level of those existing resources, researchers and communicators can have insight when working to fill the needs of agricultural business owners navigating COVID-19 impacts. Theoretical and Conceptual Framework This study was based on the conceptual framework of source credibility. The premise of source credibility is that the more credible, or trustworthy, the source, the more the receiver of information is to trust and put to use content (Kumkale et al., 2010; Pornpitakpan, 2004; Sternthal et al., 1978). The element of trustworthiness refers to the receiver’s perception of the likelihood the information the source is sharing is true (Hovland et al., 1953). There are many characteristics that can be measured or observed to substantiate whether an online source is credible, such as display of credentials of authors, whether the author represents a reputable organization, date of last update, references for the source’s data, and source funding (Hawkins, 1999). For example, Burbules (2001) notes specific markers of online credible sources as quality layouts and visually appealing sites, addressing the URL or associated emails for either academic or commercial sources, interpreting whether or not the source takes an authoritative stance or a stance with obvious bias, and if the website has been updated or Zagonel et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v2i3.145 41 revised frequently. For the purpose of this study, only one factor related to credibility, authoring source, will be measured. Credibility criteria used in assessing information have been studied within a wide range of disciplines, notably library and information science, marketing, communications, and health sciences (Liu, 2004). According to past source credibility literature, consumers of information tended to perceive authority when information appears “scholarly” (Rieh, 2001). Moreover, perceptions of high authority are found to be associated with academic institutions and government agencies with low authority to commercial sources (Rieh, 2001). Credible sources also link to themselves in the website and include complete reference lists (Burbules, 2001). Although source credibility has not been specifically applied to resources related to mitigating the impacts of COVID-19 on the food supply chain, source credibility has been applied to other areas in agricultural communications. It is known that agricultural communicators should consider the trustworthiness and perceived expertise of sources, such as representatives from regulatory agencies, educational institutions, members of the agriculture sector, or environmental organizations, when developing messages (Lamm et al., 2016). Thus, messages delivered by an individual that is perceived as a credible source may have a more meaningful effect than messages delivered by a source that lacks traits of credibility (Telg et al., 2012). It is suggested that website source credibility often has the same effect. Purpose The overall purpose of this study was to explore an area of literature otherwise unknown and to describe resources available to help enable agricultural business owners to make better informed decisions based on credible, trustworthy resources during the COVID-19 pandemic. Thus, helping create alternative methods of buying and selling and lessening the impacts from social behavior changes. The conceptual framework of source credibility guided this study in addressing the following research objectives: RO1) Describe resources available related to the impacts of COVID-19 on the food supply chain. RO2) Determine the credibility, in terms of authoring source, of available resources that aim to help mitigate impacts of food supply chain disruptions from COVID-19. Methods This study employed a quantitative content and textual analysis approach using purposive sampling to gather resources in three pre-determined ways. Quantitative content analyses are often used to systematically measure and make sense of materials (Berelson, 1952). The first way the sample was gathered was by using a United States Department of Agriculture (USDA) website that aggregated resources related to the topic of COVID-19 food supply chain disruptions. As of February 2021, this site included 171 resources. Next, the research team developed a list of search terms that could be used to locate resources related to this topic. Each search included the term COVID-19 plus an additional word or phrase, such as farm resources, selling online, buying from farms, local food, food supply chain, regional food, emergency food, curbside pickup, food processing and marketing, food and farm closure, farm legal support, farmworkers, ghost kitchens, food hubs, meatpacking, food boxes, and grocery Zagonel et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v2i3.145 42 suppliers. These search terms resulted in 579 novel resources. Resources included in the sample were gathered from the first three pages of the search results, a method based on user search behavior (Law, 2019) and previous work (Baker, McLeod-Morin, Kent, et al., 2020). The third way the sample was collected was through emails with resource links sent to the research team, which resulted in 29 resources. Duplicates were removed in the final stage. The final sample included 779 unique resources. To describe the sample and assess credibility of resources, researchers developed a codebook, a recommended practice in content analyses (Krippendorff, 2013). This codebook provided guidance to the two coders on how to systematically categorize the resources (variable), in terms of credibility (measurement). The coders conducted a priori coding, where the variables were assessed, and measurement was pre-established based upon literature and framework (Stemler, 2000). The codebook was created by examining characteristics that make a source credible (Hawkins, 1999) and applying these elements to topic areas in this study. For the variable credibility, a scale of 1 (lowest) to 3 (highest) credibility. A resource was coded as a one if there was no clear association/author or it was unknown. Resources were coded as two if the author was a commodity association, co-op, business/industry organization, nonprofit, or a news source. Resources coded as three were considered of the highest credibility and consisted of authors who were university scientists, USDA scientists, university programs, or an Extension system. The codebook was reviewed by a panel of experts for face content and validity, both of which lend credibility to the findings of this study (Krippendorff, 2013). While coding the resources for credibility, the coders also added each resource’s introduction or about information to the code sheet. This data was analyzed using a textual analysis software (Anthony, 2021) to describe the resources in the sample by identifying most frequently used terms in the texts. Two coders, a research assistant and an undergraduate researcher, were trained on how to use and interpret the codebook and were supervised by the lead principal investigator. Prior to coding the full data set, inter-rater reliability was assessed using 20% of the sample and Cohen’s Kappa (Cohen, 1960). This test is used to measure the percent of agreement between coders, while also considering coders may agree on measurement by chance (Stemler, 2000). According to Landis and Koch (1977), a Kappa score of .8 is considered substantial and deemed acceptable for social science research. The credibility variable, in this study, was assessed at .8, Cohen’s assumptions were satisfied (Cohen, 1960). Three other variables were not considered reliable at the .8 level and were removed from the study. After assessing reliability, the full data set was split between the two coders and coded during the spring of 2021. Disagreements on reliability coding were resolved by random assignment (Lacy et al., 2015), and data were analyzed for descriptive statistics using IBM SPSS 26. Zagonel et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v2i3.145 43 Findings RO1 Description of Available Resources Researchers identified 779 unique resources during the three-part sampling procedure described in the methods. To understand the content contained within the resources, textual analysis of the resources’ introduction and about paragraphs were used. In all, the analysis found 401 individual terms. Singular and plural forms of the same word, as well as differing tenses of the same word were grouped together as one word. In these scenarios, the frequency count of both words were added together. The data analysis software uses a series of three separate dictionaries of words in the English language and excludes overly common words like of, is, the, etc. When plotted on a line chart, the term frequencies leveled after the top 36 words. Top words that were found to be used over 100 times in resources’ introduction and about paragraphs, in descending order, were: farmer, farmers; resource, resources; farm, farms; market, markets; business, businesses; local, locals, locally; health, healthier, healthy; safe, safely, safety; supply, supplies, supplying; agriculture, agricultural; and chain, chains (see Table 1). Zagonel et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v2i3.145 44 Table 1 Top 36 Words Used in Introduction and/or About Paragraphs of Resources Related to COVID-19 Disruptions on the Food Supply Chain (n = 36) Word Frequency of Word Use % of Frequency Out of Total Frequencies farmer, farmers 186 2.60 resource, resources 159 2.22 farm, farms 155 2.16 market, markets 153 2.14 business, businesses 152 2.12 local, locals, locally 128 1.79 health, healthier, healthy 122 1.70 safe, safely, safety 118 1.65 supply, supplies, supplying 106 1.48 agriculture, agricultural 104 1.45 chain, chains 103 1.44 industry, industries 86 1.20 system, systems 85 1.19 consumer, consumers 84 1.17 state, states 80 1.12 community, communities 79 1.10 new 79 1.10 provide, provides, providing 73 1.02 information 73 1.02 program, programs 69 .96 worker, workers 67 .94 restaurant, restaurants 65 .91 work, working 64 .89 support, supported, supporting 61 .85 person, personal, people 60 .84 home 59 .82 country, countries 52 .73 spread, spreads, spreading 52 .73 national, nationwide 48 .67 school, schools 48 .67 operation, operations 47 .66 customer, customers 46 .64 service, services 45 .63 delivery, deliveries, delivering, delivered 43 .60 department 43 .60 people 43 .60 Zagonel et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v2i3.145 45 RO2 Authoring Source Credibility of Resources To determine the credibility, in terms of authoring source, of the resources, each resource was coded for its level of credibility based on the codebook. Out of all 779 resources in the sample, two thirds of the resources (f = 514, 66%) were at a mid-level of credibility (2 on a 3-point scale). Example sources from this level would be agricultural commodity associations, agricultural cooperatives, business/industry organizations, nonprofits, and online or print news sources. Approximately one third (f = 251, 32.2%) of all sources were considered to have the highest level of credibility. These sources were university scientists, USDA scientists, universities, university affiliated programs, governmental organizations, or Extension systems. Finally, nearly 2% (f = 14) of all resources exhibited no clear association or author on the resource. Credibility levels can be seen in Table 2. Table 2 Source Credibility Frequencies and Percentages of Resources Related to COVID-19 Disruptions on the Food Supply Chain (n = 779) Level of Credibility f % Level 2 Commodity association, cooperative, business/industry organization, nonprofit, online or print news source 514 66 Level 3 University scientist, United States Department of Agriculture scientist, university, university program, governmental organization, or Extension system 251 32.2 Level 1 No clear association/unknown 14 1.8 Note. Scale: 1 = least credible, 2 = mid-level credibility, 3 = highest level of credibility Conclusions, Discussion, and Recommendations Limitations specific to this study are that this sample is a snapshot of the resources available at one moment in time. Moreover, this was exploratory research into this specific topic area; therefore, the codebook used was researcher developed, which may include bias in interpretation. This was mitigated through inter-rater reliability but should still be noted. Additionally, some resources may have been missed in the development of the sample. Results from this study spotlight that there is no shortage of resources available to understand food supply changes and opportunities related to COVID-19 with 779 resources available online related to this topic. Textual analysis from this study showed that general production terms, such as farm(s), farmer(s), and agriculture were highly used among resources. Moreover, terms related to health and safety during COVID-19 were also frequently used, which indicates many resources were following and recommending safe protocols for business operations. Frequency of terms also indicated that of alternative food systems could be being explored by the recurring use of the term local(lly). Topic areas that had less frequently used terms, and Zagonel et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v2i3.145 46 therefore less resources available, were restaurants, schools, and deliveries. Though these terms made the top 36 list of terms in the textual analysis, these make up a far less of the percentage of the total frequency for the top words. For entities creating, organizing, or promoting materials to help business owners navigate COVID-19 disruptions to the food supply chain, it is recommended to include resources for less traditional food supply chain partners and processes, such as restaurants, schools, and deliveries. Finally, terms like Extension and research- or science-based were not frequently used. Thus, this could serve as an area of opportunity for Extension systems to publish resources and be considered a credible source in their communities during COVID-19 and future crises involving food supply disruption. Similarly, in terms of source credibility, the majority of the resources had a mid-level credibility from sources like industry, commodity, or grower organizations, nonprofits, online or print news sources, agricultural cooperatives, etc. This work demonstrates that universities and Extension systems have provided education in this area but not near to the same extent as industry. Part of this is likely due to time constraints and perhaps a focus of quality over quantity, but it does appear there is room for universities and Extension systems to increase available resources on understanding food supply changes and opportunities to sell and buy crops in alternative methods due to COVID-19. Because universities and Extension systems are typically seen as more credible sources, it is possible that people would be more likely to retain and implement information provided by these sources (Kumkale et al., 2010; Pornpitakpan, 2004; Rieh, 2001; Sternthal et al., 1978). Given the number of resources found for this study, it could be assumed that delving through credible and useful resources, as an agricultural business owner, could be overwhelming. Researchers recommend that university and Extension scientists develop and make available resources on these topics for COVID-19 and future crises, while keeping in mind specific needs or circumstances of different communities. Resources produced from both industry and academia should follow best practices of credible sources, like linking previous related information from themselves, creating visually appealing graphics and layouts on quality websites, providing contact information for websites, and updating information on the sites frequently (Burbules, 2001; Hawkins, 1999). In addition, Extension and academia should look for unique opportunities to disseminate this important information. Possible ideas include collaborating with industry and commodity partners by providing fact sheets and/or infographics that can be shared through traditional media outlets and on social media. Even developing op-ed and blog pieces that can be shared among specific target audiences would be beneficial for getting specific credible information to unique audiences. It is also recommended that no stone be left unturned when marketing and cross-promoting these resources to ensure agricultural business owners see credible resources multiple times, thus adding to the credibility of the source. Since published journal articles can be time restricting, Extension and researchers could develop research-based white papers that can be easily accessed by different organizations and used a source for news stories surrounding the crisis. To aid in future crises and disruptions to the food supply chain, it is recommended that researchers examine information sources from producers, suppliers, business owners, and consumers alike. It would also be wise conduct a needs assessment with agricultural business owners that identifies Zagonel et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v2i3.145 47 preferential communication methods as a one component of the research. Both of these efforts could further help in connecting the right resources with the right people. Furthermore, thinking about the bigger picture of future disruptions to the food supply chain, it could be useful to identify opinion leaders among communities and industries to implement and utilize best practices prescribed by credible sources. Lastly, communicators and researchers should work to build relationships with agricultural business owners, whether that is online or in real life, prior to disasters to help audiences easily identify credible sources and seek information there first. Finally, it is interesting to note that 2% of the resources had no clear association. Although this is a small percentage, it provides a unique opportunity for agricultural communicators to understand the importance of providing sources in their products. This is especially crucial during a crisis when source credibility offers a way to swim through the sea of misinformation. Acknowledgements This work was supported by the “Lessons from COVID-19: Positioning Regional Food Supply Chains for Future Pandemics, Natural Disasters and Human-made Crises” project and is funded by the Agriculture and Food Research Initiative, grant no. 2020-68006-33037, from the USDA National Institute of Food and Agriculture. References Anthony, L. (2021). AntWordProfiler (Version 1.5.1) [Computer Software]. Waseda University. https://www.laurenceanthony.net/software Baker, L., Kandzer, M., Rampold, S., Chiarelli, C., Peterson, H., & McLeod-Morin, A. (2020). Agriculture and natural resources business owners economic and communication concerns early in the COVID-19 pandemic. Advancements in Agricultural Development, 1(3), 95–110. https://doi.org/10.37433/aad.v1i3.83 Baker, L., McLeod-Morin, A., Kent, K., & Lindsey, A. (2020). No online information outbreak: A quantitative content analysis of the CDC and USDA websites for available information on zoonotic disease. Advancements in Agricultural Development, 1(1), 25–37. https://doi.org/10.37433/aad.v1i1.19 Berelson, B. (1952). Content analysis in communication research. The ANNALS of the American Academy of Political and Social Science, 283(1), 197-198. https://doi.org/10.1177/000271625228300135 Burbules, N. C. (2001). Paradoxes of the web: The ethical dimensions of credibility. Library Trends, 49(3), 441–453. https://www.ideals.illinois.edu/bitstream/handle/2142/8352/librarytrendsv49i3f_opt.pdf Zagonel et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v2i3.145 48 Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. https://doi.org/10.1177/001316446002000104 Duvall, Z. (2020). Lessons from COVID-19. American Farm Bureau Federation. https://www.fb.org/viewpoints/lessons-from-covid-19 Hawkins, D. T. (1999). What is credible information? Online, 23(5), 86–89. Hovland, C., Janis, I., & Kelley, H. (1953). Communication and persuasion. Yale University Press. Krippendorf, K. (2013). Content analysis: An introduction to its methodology. SAGE. Kumkale, G. T., Albarracín, D., & Seignourel, P. J. (2010). The effects of source credibility in the presence or absence of prior attitudes: Implications for the design of persuasive communication campaigns. Journal of Applied Social Psychology, 40(6), 1325–1356. https://doi.org/10.1111/j.1559-1816.2010.00620.x Lacy, S., Watson, B. R., Riffe, D., & Lovejoy, J. (2015). Issues and best practices in content analysis. Journalism & Mass Communication Quarterly, 92(4) 791–811. https://doi.org/10.1177/1077699015607338 Lamm, A. J., Owens, C. T., Telg, R., Lamm, K. W. (2016). Influence of source credibility on agricultural water use communication. Journal of Applied Communications, 100(3), 121– 132. https://doi.org/10.4148/1051-0834.1235 Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. http://dx.doi.org/10.2307/2529310 Law, T. J. (2019, November 22). Google SERPs: Features and how to improve your ranking in 2019. Oberlo. https://www.oberlo.com/blog/serp-google-search-engine-results Lioutas, E. D., & Charatsari, C. (2021). Enhancing the ability of agriculture to cope with major crises or disasters: What the experience of COVID-19 teaches us. Agricultural Systems, 187, Article 103023. https://doi.org/10.1016/j.agsy.2020.103023 Liu, Z. (2004). Perceptions of credibility of scholarly information on the web. Information Processing & Management, 40(6), 1027–1038. https://doi.org/10.1016/S0306- 4573(03)00064-5 Nakat, Z., & Bou-Mitri, C. (2021). COVID-19 and the food industry: Readiness assessment. Food Control, 121, Article 107661. https://doi.org/10.1016/j.foodcont.2020.107661 Narine, L., & Meier, C. (2020). Responding in a time of crisis: Assessing extension efforts during COVID-19. Advancements in Agricultural Development, 1(2), 12–23. https://doi.org/10.37433/aad.v1i2.35 Zagonel et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v2i3.145 49 Nelson, M. (2020). Keeping rural communities connected while socially distanced. American Farm Bureau Federation. https://www.fb.org/market-intel/keeping-rural-communities- connected-while-socially-distanced Pauchant, T. C., & Mitroff, I. I. (1990). Crisis management: Managing paradox in a chaotic world. Technological Forecasting and Social Change, 38(2), 117–134. https://doi.org/10.1016/0040-1625(90)90034-S Pornpitakpan, C. (2004). The persuasiveness of source credibility: A critical review of five decades’ evidence. Journal of Applied Social Psychology, 34(2), 243-281. https://doi.org/10.1111/j.1559-1816.2004.tb02547.x Rieh, S. Y. (2001). Judgment of information quality and cognitive authority in the Web. Journal of the American Society for Information Science and Technology, 53(2), 145–161. https://doi.org/10.1002/asi.10017 Sahin, O., Salim, H., Suprun, E., Richards, R., MacAskill, S., Heilgeist, S., Rutherford, S., Stewart, R. A., & Beal, C. D. (2020). Developing a preliminary causal loop diagram for understanding the wicked complexity of the COVID-19 pandemic. Systems, 8(2), 20. https://doi.org/10.3390/systems8020020 Stemler, S. (2000). An overview of content analysis. Practical Assessment, Research, and Evaluation, 7(17), 1–6. https://doi.org/10.7275/z6fm-2e34 Sternthal, B., Dholakia, R., & Leavitt, C. (1978). The persuasive effect of source credibility: Tests of cognitive response. Journal of Consumer Research, 4(4), 252– 260. https://doi.org/10.1086/208704 Telg, R., Irani, T., Monaghan, P., Chiarelli, C., Scicchitano, M., & Johns, T. (2012). Preferred information channels and source trustworthiness: Assessing communication methods used in Florida’s battle against citrus greening. Journal of Applied Communications, 96(1), 1–12. https://doi.org/10.4148/1051-0834.1147 Thilmany, D., Jablonski, B., Angelo, B., Low, S., & Tropp, D. (2020). Mitigating immediate harmful impacts of COVID-19 on Colorado farms and ranches selling through local and regional food markets. Regional Economic Development Institute https://mountainscholar.org/handle/10217/217134 © 2021 by authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).