Bunch_2020_UF_International_Educator Dossou Kpanou et al. Advancements in Agricultural Development Volume 1, Issue 3, 2020 agdevresearch.org 1. Bidossessi M. G. Dossou Kpanou, Graduate Research Associate, University of Georgia, 203 Lumpkin House, 145 Cedar Street, Athens, GA, 30602, bd8245@uga.edu, https://orcid.org/0000-0002-7969-0811 2. Kathleen D. Kelsey, Professor and Director of the Impact Evaluation Unit, University of Georgia, 203 Lumpkin House, 145 Cedar Street, Athens, GA, 30602, kdk@uga.edu, https://orcid.org/0000-0001-9683-6993 3. Kyle L. Bower, Postdoctoral Research Associate, University of Georgia, 203 Lumpkin House, 145 Cedar Street, Athens, GA, 30602, kbower@uga.edu, https://orcid.org/0000-0001-6043-356X 42 Evaluation of a Federally Funded Research Network Using Social Network Analysis B. Dossou Kpanou 1, K. Kelsey2, K. Bower3, Abstract United States federal agencies fund research to promote discovery and innovation. Most agencies require collaboration because teams promote productivity to a greater degree than singular researchers. However, the functionality and productivity of collaboration is poorly understood. The purpose of this study was to evaluate the collaborative structure of a federally funded entomology research team to determine the characteristics of the network structure and its impact on research collaboration using social network analysis (SNA) methodology. An online survey and interviews were used to collect data. The theories of social network, strong and weak ties, and scientific collaboration were employed to determine the degree of collaboration among team members. We found a low-density pattern of collaboration that was associated with: (a) a centralized pattern, (b) the presence of sub-teams functioning like sub-networks, and (c) the presence of less interactive members. Our results confirm that the SNA approach was useful for evaluating network collaboration with innovative indicators to assess the dynamics of scientific collaboration. The study was limited by non-response. Future research should focus on collecting SNA data longitudinally of the whole network to determine how networking structure and benefits evolves over time, and how strong and weak ties impact scientific discovery. Keywords Scientific collaboration, network connectivity, evaluation. Dossou Kpanou et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i3.65 43 Introduction and Problem Statement United States (U.S.) federal agencies fund research to promote discovery and innovation. Most agencies require teams to collaborate on successful awards because they promote productivity to a greater degree than research projects executed by a single investigator (Contandriopoulos et al., 2018; Lee & Bozeman, 2005). Academic collaboration is also important as it fosters scientific discovery and generates knowledge (Katz & Martin, 1997; Lee & Bozeman, 2005). Accordingly, agencies such as the U.S. Department of Agriculture (USDA, 2019) and the National Institute of Food and Agriculture (NIFA) seek to enhance the productivity and efficiency of scientific research by requiring collaboration as stated in their request for proposals (Katz & Martin, 1997; Lee & Bozeman, 2005). The development of new scientific networks, or the continuation of existing collaborations, depends on the success and productivity of previous collaboration. Researchers who have worked together in the past are likely to continue their partnerships on future scientific projects (Contandriopoulos et al., 2018), while research networks are strengthened by their ability to secure additional funding having demonstrated past success. Despite the plethora of evidence supporting scientific effectiveness among collaborative research teams, their structure and functionality has not been adequately evaluated in the literature. Social network analysis (SNA) is a viable methodological approach that provides researchers with scientific tools to evaluate the structure of collaboration within networks (Borgatti et al., 2018). SNA enabled our exploration of emergent characteristics of a federally funded research team focused on solving an invasive pest problem and provided an opportunity to understand the functionality of this network. Furthermore, the evaluation of ongoing scientific networking provided evidence needed to improve research productivity and future collaboration among the team. Theoretical and Conceptual Framework This study integrated principles from social network theory (Borgatti et al., 2009 & 2018), strong and weak ties theory (Granovetter, 1973; 1983; Rademacher & Wang, 2014), and scientific collaboration theory (Olson et al., 2008) to develop a comprehensive approach for describing a social network originated for the purpose of executing a federally funded research project to address a major pest infestation damaging fruits in the U.S. Social network theory (SNT) explains the essence of interactions between members of a network and emphasizes the study of network characteristics, configurations, and architectural features (Borgatti et al., 2018). Additionally, SNT moves beyond the attributes of individuals to emphasize the relationships that form within a network by mapping connections (Schmidt, 2007) between team members, or ties, that constitute the channels for exchanging resources such as knowledge, funding, and access to infrastructure (Borgatti et al., 2018). The exchange of resources is strongly influenced by a member’s position within the network and is vital for Dossou Kpanou et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i3.65 44 accessing benefits associated with networks (Hansen, 2009). Furthermore, the sociometry of social networks provides strong indicators of productivity (Hansen, 2009) because it gives insight into the collaboration structure of relationships among members who are involved in creating, disseminating, and using knowledge that adds to team functionality (Dunn, 1983). Strong and weak ties theory guided our analysis of network connectivity and information flow (Granovetter,1973, 1983; Rademacher & Wang, 2014). Strong ties within a network suggest frequent interrelations that assume better collaboration within a network (Borgatti, 2018; Granovetter, 1973). Alternatively, weak ties are characterized by distant and infrequent interrelations between members of a network, suggesting low reciprocity among members (Granovetter, 1973; Rademacher & Wang, 2014). While weak ties appear less advantageous, they benefit a network by providing a bridge between two unknown, or weakly connected, members (Leij & Goyal, 2011). Scientific collaboration theory is underpinned by communication theory (Olson et al., 2008; Olson & Olson, 2000; Sonnenwald, 2007). Research indicates that remote scientific contributors communicate less frequently than contributors working at the same institution (Ding et al., 1998; Katz, 1994; Olson et al., 2008), which leads to difficulties in sharing knowledge and resources among collaborators. This theory helped us to understand how team members in this study communicated from geographically disparate locations, placing their ability to communicate at a disadvantage. Olson et al. developed the theory of remote scientific collaboration, which recognized digital communication as an effective way for remote scientific researchers to exchange information, data, ideas, and results. Their theory indicated that communication technologies such as videoconferencing, e-mail, and instant messaging helped to circumvent the geographical distance by allowing ongoing conversations among members (Ding et al., 1998; Katz, 1994). Olson and Olson (2000) noted that despite access to advanced communication technologies, remote collaboration remains challenging. Purpose The purpose of the study was to evaluate the collaborative structure of a federally funded scientific research network. Accordingly, the primary research question was to what extent did the current characteristics of the federally funded research network structure impact the research collaboration? This question guided our evaluative inquiry into how the research project fostered transdisciplinary collaboration and knowledge sharing among team members. Methods The study was conducted through a sequential mixed-methods evaluation design (Mertens, 2018). Our approach was exploratory with a stronger emphasis placed on quantitative data. Qualitative data was used to triangulate the survey and observational findings (Fraenkel et al., 2012). The sequential design allowed us to identify factors impacting the cohesion of the Dossou Kpanou et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i3.65 45 network via survey and then gather explanatory data via interviews that aided our interpretation of the results. Data Collection The evaluation team began data collection with participant observations (Patton, 2001) during monthly team meetings, webinars, conferences, and email correspondence. Observational data gathered before and after the survey data added to the contextualization of relationship patterns observed in the network sociograms (Figures 1 and 2) and helped us to frame the survey questions, known as an ethnographic sandwich approach (Borgatti et al., 2018). To recruit for the survey, all team members (N = 52) including research faculty (n = 16), post- doctoral research associates (n = 7), advisory board members (n = 15), laboratory technicians (n = 7), a graduate research associate (n = 1), and undergraduate students (n = 6) were first informed of the study during monthly team meetings and then sent a formal invitation to participate via email with the survey link embedded. Two follow-up emails were sent to non- responders. When necessary, we called participants to encourage survey completion (Dillman et al., 2008). Thirty-six participants completed the self-administered survey through Qualtrics® for a 69% response rate. Purposeful sampling (Mertens & Wilson, 2012) resulted in telephone interviews with five faculty and one graduate student. The interviews were recorded with consent and lasted an average of 36 minutes each. While the student and faculty were asked similar questions, the semi-structured interview protocols differed slightly taking into consideration their specific roles. Advisory board members did not respond to our requests for interviews, so we emailed all of them requesting written responses to five open-ended questions regarding their overall interaction with the project. Three of 15 advisory board members responded by providing written responses. Data Analysis Survey data were cleaned by removing missing data. Non-respondents who were referred to by other team members were kept in the dataset to ensure completeness of the network analysis (Borgatti et al., 2018). To protect participants' privacy, all names were changed. Once the data were properly formatted, we used UCINET 6 software to design the corresponding sociograms (Figures 1 and 2) and computed the following SNA indicators: (a) density, (b) degree (including in-degree and out-degree), (c) degree centralization (including indegree-centralization and outdegree-centralization), (d) betweenness, (e) dyad reciprocity, and (f) betweenness centrality. Qualitative interview data were transcribed through an online program, Otter.ai (Lang, 2020). Transcripts and observation notes were then uploaded into ATLAS.ti 8 for Windows, where the data were stored, managed, and analyzed (Friese, 2019). The coding process focused on acquiring meaning from interviewees’ perspectives by identifying significant quotations related to collaboration within the data (Linneberg & Korsgaard, 2019). We performed line-by-line Dossou Kpanou et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i3.65 46 coding by using the group code collaboration and two main codes, valuing collaboration and barriers/difficulties to collaboration. Findings Overall, the network had a low-density value, which at first glance implies that it was less connected and less cohesive than a high-density network. However, through our mixed-method analysis we identified three explanatory factors which contributed to a nuanced understanding of the low-density network, including: (a) a centralized network pattern, (b) the presence of sub-teams functioning like sub-networks, and (c) the presence of less-interactive members. We first describe the a less connected network pattern using the SNA metric of density and present two sociograms to illustrate this finding (Figures 1 and 2). Next, we explore the centralized pattern of the network and the presence of sub-teams using specific SNA indicators and the sociograms. Finally, we introduce qualitative data to explain how the less-interactive members contributed to reducing the overall network density. The Less Connected Structure of the Research Network Most of the team members had previously collaborated on a federally funded project, therefore, it was necessary to examine the structure of the current network against a baseline. To obtain a baseline, the question, “Who have you worked with prior to this project?” was included in the survey. Results from this question made it possible to assess changes in interactions and interconnections within the team between two time periods. Table 1 compares the indicators characterizing the structure of the network at baseline and at the time of data collection. Findings revealed that the baseline had a density of 13.3% while that of the current network was 18.2%, suggesting that the overall professional connections within the network evolved over time but remained relatively low. Table 1 Comparison of Indicators Characterizing the Structure of the Network Indicators Network Baseline At the time of data collection Density 0.133 0.182 Number of ties (Degree) 326 445 Degree Centralization 0.287 0.776 In-centralization 0.323 0.294 Out-centralization 0.261 0.835 Dyad Reciprocity 0.374 0.422 While the network density had increased and was visually perceptible when comparing the two sociograms (Figures 1 and 2), most of the members who were peripheral at baseline remained peripheral, thus, maintaining the low density of the whole network. The peripheral members Dossou Kpanou et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i3.65 47 were mainly advisory board members (names starting with letter A), students (names starting with letter S), and laboratory technicians (names starting with letter L). Generally, the indicators of the current network showed improvement in the number of ties, degree centralization, and dyad reciprocity between the baseline and at the time of the study. On average, at the time of data collection, each member shared information and data with nine other members of the network, higher than 6.5 at baseline. In addition, when compared to its initial structure (Figure 1), Figure 2 visually supports the findings indicating that the current research network has a more connected and centralized structure. Figure 1 Sociogram of Members’ Professional Acquaintances Prior to the Project (Baseline) Note. Names starting with I are the principal investigators (red), names starting with C are the co-principal investigators (pink), names starting with P are postdocs (orange), names starting with A are the advisory board members (yellow), names starting with L are laboratory staff (green triangle), names starting with S are students (green square), and black circles are non- respondents. Dossou Kpanou et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i3.65 48 Figure 2 Sociogram of Members’ Professional Acquaintances at the time of Study Note. Names starting with I are the principal investigators (red), names starting with C are the co-principal investigators (pink), names starting with P are postdocs (orange), names starting with A are the advisory board members (yellow), names starting with L are laboratory staff (green triangle), names starting with S are students (green square), and black circles are non- respondents. The Network Centralization as a Factor of the Network Low Connectedness The structure of the research network, characterized by a centralized pattern with few central members and several peripheral members, explains the low connectedness of the network. Initially, the baseline network (Figure 1) exhibited a structure with relatively low degree centralization, 0.287. At the time of data collection, the network became more centralized, which is substantiated by a relatively high degree centralization value of 0.776. For instance, central members, namely Ivan (the project director), Ismael, Peter, and Alice (Figure 2) played a key role in disseminating information and data within the network. In particular, the findings revealed that with a degree centrality of 65 and a betweenness centrality score of 407.19, Ivan was the most central member of the research network. Additionally, out-degree and in-degree centralization values supported the centralization pattern of the network around a few central members. The out-centralization, which is the total number of connections going out from all members, was 0.835 (Table 1), suggesting that there were high outgoing interactions compared to the total possible outgoing interactions. On the other hand, the in-degree centralization, which is the total number of connections being received by members, was 0.294 for the whole network, meaning there were fewer incoming interactions compared to the total possible incoming interactions. Both changes were significant at the 0.05 level (p-value = 0.002 for out-degree, p-value = 0.000 for in-degree). Dossou Kpanou et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i3.65 49 The Sub-Teams as a Factor Affecting the Network Connectivity The dyad reciprocity was 0.422, which means that from all possible relationships between all pairs of members 42% of the connections occurred. This indicates a relatively low reciprocity among the network and supports the low connectivity finding of the whole network. Furthermore, we explored sub-teams’ characteristics from four sub-teams (Table 2). While we did identify more than four sub-teams within the network, we were unable to detect significant results due to small sub-team size (3 or fewer members), low interaction within the sub-teams, and low response rates within the sub-teams. Table 2 Characteristics of Sub-Teams Sub-Teams # of Members Density Dyad Reciprocity Sub-team 1 4 0.91 0.83 Sub-team 2 6 0.57 0.70 Sub-team 3 5 0.55 0.37 Sub-team 4 8 0.83 0.67 Note. The four sub-teams had at least four members each. The density of each sub-network was higher than the whole network density value (18%). Also, three of the four sub-teams had higher dyad reciprocity than the whole network value (42%), indicating that networking was stronger within sub-teams than the whole network. For example, sub-team 1 and sub-team 4 densities regarding information sharing were respectively 91% and 83%, suggesting well-connected sub-teams with a high flow of information supported by their relatively high dyad reciprocity of 83% and 67%. Also, apart from sub-team 2, all other sub-teams reported having frequent interactions substantiated by high levels of internal interactions (Table 3). Table 3 Frequency of Interaction within Sub-Teams Sub-teams # of Members Rarely Sometimes Often Very often Most often Sub-team 1 4 2 1 3 0 6 Sub-team 2 6 10 5 0 2 4 Sub-team 3 5 0 0 2 9 5 Sub-team 4 8 0 21 17 15 3 Note. The four sub-teams had at least 4 members. Dossou Kpanou et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i3.65 50 Low connectivity within sub-teams (e.g., sub-team 3) and less frequent interactions (e.g., sub- team 2) is explained by the absence of relationships between some sub-team members and by the few reciprocal relationships between them. This impacted the connectivity of the whole network. Participant observations and interviews further supported this finding. For instance, during meetings and conferences we observed self-distinction made between the university research teams. i.e. the researchers identified themselves based on their employer’s affiliation versus members of the whole network (Entomology team). Further, interview data supported this pattern of collaboration. For example, Charlie (faculty) stated, “truth be told, we have been working more independently than we did before. We are working more on being parallel with the [other sub] teams”. Faculty reported that in practice, because their collaboration encompassed independent research, the sub-teams worked separately from the whole network. Although we cannot determine if this distinction will affect the research outputs over time, it explains the emerging pattern of interactions with more intense collaborations within the sub-teams. The Meaning of Less Interactive Members The qualitative data provided insights into the nature of interactions within the whole network and explained the effect of less interactive members on low connectivity by contextualizing interactions with each other and within sub-teams. While sub-team and group (faculty, student, advisory board) members’ interactions with each other were infrequent, they were productive. Student activity was most important to bolstering research outputs. Faculty reported that their student advisees, both graduate and undergraduate, were deeply involved in the research activities. Faculty were most connected to their student advisees. However, interactions between students and faculty beyond their sub-teams was limited and had the effect of decreasing the whole network density value. Sub-team 4 worked on the economic aspect of the project, differing from the other sub-teams who were involved in entomology experiments. Sub-team 4 interactions with the whole network were limited because of their unique mission. Lucas, graduate student in sub-team 4, reported that collaborations with other sub-teams would occur later in the project. He said, “A lot of their lab studies are not going to be directly applicable to our economic stuff. It is going to be over the next year that [we are] going to be collaborating a lot more with [the other sub- teams].” Lucas added that interactions between his sub-team and the other sub-teams would determine who they would most likely work with in the future. Advisory board members’ infrequent interactions with faculty also lowered the whole network density value. Participant observations noted low attendance of advisory members at conferences and webinars. Faculty said that advisory members were more interested in receiving information regarding pest solutions than discussing emerging research findings, which was the focus of most of the team meetings and webinars. This finding was also triangulated by advisory board members in their responses to the questionnaire. Of the three who responded, two wrote they only interacted with the research sub-team located in their state of residence and the third did not mention any interaction with faculty. Although advisory Dossou Kpanou et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i3.65 51 board members’ involvement in research meetings was infrequent, faculty expressed the importance of their engagement during farm demonstrations and commented on their positive contributions toward research conducted on their farms. While the mostly unilateral transfer of information from faculty to students and advisory board members led to low density values, knowledge and resources were shared bilaterally to promote the advancement of research that benefited the whole network. Conclusions, Discussion, and Recommendations Overall, the whole network demonstrated a relative low-density value, which suggests a weakly connected network. According to Borgatti et al. (2018), the judgment of high or low connectivity as measured by network density depends on the context. To better understand the relatively low connectivity of the network, we explored the centralized pattern of the network, the presence of sub-teams, and the meaning of less interactive members within the network. We expected the whole network to have a high-density value that theoretically supports a well- connected network because it was a closed network focused on solving an invasive pest issue (Borgatti et al., 2018). However, the overall low density of the network was associated with the centralized pattern of the network. The central members, including the project director (Ivan), sent information and data to members more frequently than they received it. Thus, they facilitated the interconnection between sub-teams, resulting in a positive influence on collaboration, building on weak ties theory (Zamzami & Schiffauerova, 2017) that suggests scientists who are central within their networks have a positive impact on the flow of information across sub-teams who may not otherwise be connected in the same whole network. The low connectedness value of the whole network can also be explained by the presence of sub-teams, which functioned like mini networks within the whole network. Sub-team internal interactions were governed by pre-assigned research questions at project initiation. Therefore, there was little need for sub-teams to communicate frequently with other sub-teams to accomplish their work. This finding is consistent with Hoang et al. (2019) who concluded that working on the same research topics is an essential factor that encourages interaction between faculty. Given the importance that funding agencies, especially NIFA, place on promoting collaborative research, we recommend greater collaboration between sub-teams by creating overlapping research questions that require input from the whole network (Contandriopoulos et al., 2018; Defazio et al., 2009). Consistent with the literature regarding the value of cohesion inherent among weakly tied members (Granovetter, 1973, 1983), our finding suggests a relatively strong collaboration among the whole network despite its relatively low connectivity value. While students and advisory board members did not regularly interact with faculty, they contributed significantly to the overall research agenda. Therefore, we conclude that the quality of a scientific Dossou Kpanou et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i3.65 52 collaboration between and among sub-team members did promote the advancement of scientific discovery and fostered future collaboration, which is indicative of a strong network collaboration (Defazio et al., 2009; Hoang et al., 2019; Olson et al., 2008). We recommend that researchers working within a network include overlapping research questions and implement activities that require frequent input from all network members. Such an approach should foster better interaction between sub-networks, foster participation of all members, improve overall network connectivity, and foster future collaboration among team members. The study was limited by non-respondents. Sub-team 3 remains underexplored due to low interaction within the sub-team and the whole network. Including non-respondents in the sociograms (Figures 1 and 2) influenced the number of ties; and therefore, it is possible we may have overstated density values (Borgatti et al., 2018). Nevertheless, the contributions of this study toward better understanding the structure and function of collaborative teams highlights the contextual factors that negatively impacted the overall connectivity of the research network. Future research should focus on collecting SNA data longitudinally of the whole network to determine how the network evolves over time, and how strong and weak ties impact scientific discovery over the life of the project. Acknowledgements This research was supported by funding from the USDA-NIFA Organic Agriculture Research and Extension Initiative (Award # 2018-51300-28434). References Borgatti, S. P., Everett, M. G., & Freeman, L. C. (2002). Ucinet 6 for windows: Software for social network analysis [Computer software]. Analytic Technologies. http://www.analytictech.com/archive/ucinet.htm Borgatti, S. P., Everett, M. G., & Johnson, J. C. (2018). Analyzing social networks (2nd ed.). Sage Publications. Borgatti, S. P., Mehra, A., Brass, D. J., & Labianca, G. (2009). Network analysis in the social sciences. Science, 323(5916), 892–895. https://doi.org/10.1126/science.1165821 Contandriopoulos, D., Larouche, C., & Duhoux, A. (2018). Evaluating academic research networks. Canadian Journal of Program Evaluation, 33(1), 69–89. https://doi.org/10.3138/cjpe.42159 Dossou Kpanou et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i3.65 53 Defazio, D., Lockett, A., & Wright, M. (2009). Funding incentives, collaborative dynamics and scientific productivity: Evidence from the EU framework program. Research Policy, 38(2), 293–305. https://doi.org/10.1016/j.respol.2008.11.008 Dillman, D. A., Smyth, J. D., & Christian, L. M. (2008). Internet, mail, and mixed-mode surveys: The tailored design method (2nd ed.). Wiley & Sons. Ding, Y., Foo, S., & Chowdhury, G. (1998). A bibliometric analysis of collaboration in the field of information retrieval. International Information & Library Review, 30(4), 367–376. https://doi.org/10.1080/10572317.1998.10762484 Dunn, W. N. (1983). Social network theory. Knowledge, 4(3), 453–461. https://doi.org/10.1177/107554708300400306 Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education (8th ed.). McGraw-Hill Higher Education. Friese, S. (2019). ATLAS.ti 8 Mac user manual. https://downloads.atlasti.com/docs/manual/manual_a8_mac_en.pdf?_ga=2.31688247. 844522840.1558027933-883190210.1548880313 Granovetter, M. S. (1983). The strength of weak ties: A network theory revisited. Sociological Theory, 1, 201–233. https://doi.org/10.2307/202051 Granovetter, M. S. (1973). The strength of weak ties. American Journal of Sociology, 78(6), 1360–1380. https://doi.org/10.1086/225469 Hansen, T. (2009). Applying social network theory and analysis in the struggle for social justice. Peace Research, 41(1), 5-43. www.jstor.org/stable/23607964 Hoang, D. T., Nguyen, N. T., Tran, V. C., & Hwang, D. (2019). Research collaboration model in academic social networks. Enterprise Information Systems, 13(7-8), 1023-1045. https://doi.org/10.1080/17517575.2018.1556812 Katz, J. S. (1994). Geographical proximity and scientific collaboration. Scientometrics, 31(1), 31– 43. https://doi.org/10.1007/bf02018100 Katz, J. S., & Martin, B. R. (1997). What is research collaboration? Research Policy, 26(1), 1–18. https://doi.org/10.1016/s0048-7333(96)00917-1 Lang, S. (2020). Otter [Computer software]. http://otter.ai Lee, S., & Bozeman, B. (2005). The impact of research collaboration on scientific productivity. Social Studies of Science, 35(5), 673–702. https://doi.org/10.1177/0306312705052359 Dossou Kpanou et al. Advancements in Agricultural Development https://doi.org/10.37433/aad.v1i3.65 54 Leij, M. V. D., & Goyal, S. (2011). Strong Ties in a Small World. Review of Network Economics, 10(2), 1–42. https://doi.org/10.2202/1446-9022.1278 Linneberg, M. S., & Korsgaard, S. (2019, March 16). Coding qualitative data: A synthesis guiding the novice. https://www.researchgate.net/publication/332957319_Coding_qualitative_data_a_synt hesis_guiding_the_novice Mertens, (2018). Mixed methods design in evaluation. Sage Publication. Mertens, D. M., & Wilson, A. T. (2012). Program evaluation theory and practice: a comprehensive guide. Guilford Press. Olson, J. S., Hofer, E. C., Bos, N., Zimmerman, A., Olson, G. M., Cooney, D., & Faniel, I. (2008). A theory of remote scientific collaboration. Scientific Collaboration on the Internet, 73–97. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.8012&rep=rep1&type= pdf Olson, G. M., & Olson, J. S. (2000). Distance matters. Human–computer interaction, 15(2-3), 139–178. https://doi.org/10.1207/s15327051hci1523_4 Patton, M. Q. (2001). Qualitative research and evaluation methods. Sage Publications. Rademacher, M. A., & Wang, K. Y. (2014). Strong-tie social connections versus weak-tie social connections. Scholarship and Professional Work -Communication. 103, 1213-1216. http://digitalcommons.butler.edu/ccom_papers/103 Schmidt, A. (2007). Social network theory. Encyclopedia of Governance, 893–895. https://doi.org/10.4135/9781412952613.n503 U. S. Department of Agriculture. (2019). National Institute of Food and Agriculture FY 2018 Annual Report. https://nifa.usda.gov/sites/default/files/resource/NIFA-2018-Annual- Report.pdf Sonnenwald, D. H. (2007). Scientific collaboration. In: B. Cronin (Ed.), Annual Review of Information Science and Technology, 41, (pp. 643-681). https://www.researchgate.net/publication/220040763_Scientific_collaboration Zamzami, N., & Schiffauerova, A. (2017). The impact of individual collaborative activities on knowledge creation and transmission. Scientometrics, 111(3), 1385–1413. https://doi.org/10.1007/s11192-017-2350-x © 2020 by authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).