Pine et al The Journal of Community Informatics ISSN: 1721-4441 3 Articles Addressing Fragmentation of Health Services through Data-Driven Knowledge Co-Production within a Boundary Organization Kathleen H. Pine, Arizona State University, khpine@asu.edu Margaret Hinrichs, Arizona State University, mhinrich@asu.edu Kailey Love, Arizona State University, Kailey.love@asu.edu Michael Shafer, Arizona State University, msshafer@asu.edu George Runger, Arizona State University, George.runger@asu.edu William Riley, Arizona State University, william.j.riley@asu.edu The Journal of Community Informatics ISSN: 1721-4441 4 Addressing Fragmentation of Health Services through Data-Driven Knowledge Co-Production within a Boundary Organization Abstract Behavioral healthcare services involve multiple disconnected sectors and providers serving the same populations. Efforts to identify and address service delivery problems are hampered by fragmentation of organizations and datasets. We conducted an action research project in which we formed a boundary organization and developed a knowledge co-production process (Data- Driven Knowledge Co-Production or DDKCo) centered on collaborative data sharing and visualization. Multisector participants in Phoenix worked to access and share data and to collectively interpret the resulting integrated data through visualizations using four knowledge co-production practices: collective interaction with data, perspective taking, reflection & debrief, and iteration of visualizations. The knowledge co-production process was evaluated using qualitative methods. This research extends knowledge co-production literature by proffering collaborative data sharing and visualization as a knowledge co-production process that can extend across disconnected and disparate social groups and contributes to community informatics by highlighting the role boundary organizations can play in facilitating data sharing and data-driven problem solving between fragmented sectors. Keywords: Data Sharing; Knowledge Co-Production; Boundary Organizations; Behavioral Healthcare; Action Research Introduction The field of community informatics (CI) has long been concerned with the collection, integration, and use of data for community health services (Namatovu et al., 2017; Obeysekare et al., 2017). In the United States, behavioral health is a major issue (NAMI, 2019). While a wealth of community based behavioral health services data is available (and thus could be used to identify service problems and improve health services delivery), use of these data is hampered by the fact that individuals with behavioral health disorders are served by multiple service sectors and providers in a community (Bhugra et al., 2014). Though these providers have overlapping partial information about the populations they serve, datasets are siloed, making it hard to detect or address problems that go beyond singular organizations. Data sharing is a process that often hinges on social connections between those sharing data and tacit knowledge that allows different users of data to assess its availability and quality, along with an understanding of the conditions under which data were collected and the various limitations of datasets (Cragin et. al., 2010; Faniel & Jacobsen, 2010; Wallis et al., 2013; Zimmerman, 2008). The Journal of Community Informatics ISSN: 1721-4441 5 Still, data sharing across disconnected organizations is crucial. According to theories of knowledge, data are a first step in achieving knowledge (Ackoff, 1998). Empirical research shows that data must be taken up in practice and contextualized to be made actionable as knowledge (Alvarado-Garcia et al., 2017; Nafus, 2014; Ottinger & Zurer, 2011). Knowledge co-production in multisector collaboration is vital in alleviating complex problems (Howarth & Monasterolo, 2017) such as those present in the behavioral health care delivery system. Knowledge co-production refers to processes that iteratively unite ways of knowing and acting, including ideas, norms, practices, and discourses, leading to mutual reinforcement and reciprocal transformation of societal outcomes (Vindrola-Padros, 2018), as well as increasing the usability and value of science for society (Meadow et al., 2015). The knowledge co-production literature shows that it can be beneficial for a guiding organization to facilitate the process of knowledge co-production. However, in complex health services where there is a need for data sharing amongst multisector providers serving the same patient population, identifying a single organization to lead knowledge co-production may be difficult. An unexplored solution for organizing such multisector data sharing and knowledge co- production efforts is the formation of a boundary organization. Boundary organizations are defined as organizations that cross the boundary between science and policy and draw on the interests and knowledge of multisector stakeholders to facilitate evidence-based and socially beneficial interventions (Guston, 2001). Boundary organizations are known to have the following attributes: they involve participation by researchers and policymakers as well as professionals mediating between these groups; they provide incentives for the creation of collective artifacts; and they are accountable to both research and policy communities (Cash, 2001; Crona & Parker, 2011; Guston, 2001; White et al., 2010). Starting with a problem (i.e., the fragmentation of organizations and datasets hampering systems-level problem solving for behavioral health services) identified by stakeholders working in behavioral health services in Phoenix, Arizona, we worked alongside stakeholders to find a means to address this problem. As a result, we developed a novel process drawing on boundary organizations and the knowledge co-production literature. We extend past CI research on identifying and overcoming barriers to data use such as data literacy (Frank et al., 2016; Frank & Walker, 2016) and uptake of open data initiatives (Lupi et al., 2020) by proposing data-driven knowledge co-production (DDKCo) as a process to promote data integration and delivery of community-based behavioral health services. Through our analysis of semi-structured interviews and participant observations of stakeholder activities, we answer the following research questions: how do boundary organizations promote data sharing and knowledge co-production in a community health informatics initiative? What are the situated processes through which knowledge co-production occurs between multisector participants? Finally, what are the outcomes produced by facilitating data sharing and knowledge co-production among multisector participants? Introduction Health Data Sharing The Journal of Community Informatics ISSN: 1721-4441 6 Research on data sharing shows that socio-cultural aspects of data sharing are crucial in data- driven initiatives. Trust and relationships are of paramount importance in scientific data sharing. Faniel & Jacobsen (2010) described key factors that impact whether scientists share data and reuse others’ shared data. These factors include: 1) knowing what data are available; 2) understanding the circumstances under which data were collected well enough to use it properly; and 3) being able to accurately assess the quality of data and value for their own intents and purposes. Because researchers draw on their own experience in making determinations about data quality and data utility for their own needs, much of this guiding knowledge is informal (Zimmerman, 2008). Moreover, relationships are a crucial part of data sharing both outside and inside of formal organizations (Alvarado-Garcia et al., 2017; Cragin et al., 2010; Wallis et al., 2013). Studies of organizational and patient data sharing in the healthcare domain are quite limited (Rowhani-Farid, Allen, and Barnett, 2017), despite a growing body of literature that focuses on promoting interoperability (Kasthurirathne et al., 2015; Liyanage et al., 2015) and the integration of disparate healthcare datasets (Gay & Leijdekkers, 2015; Marceglia et al., 2015) to improve health services research and practice. These studies have indicated a strong belief that sharing health services data would lead to improvements in healthcare delivery. However, there is a pressing need for studies of data sharing across healthcare sectors and organizations. Similarly, calls for data sharing in the public health field demonstrate immense interest in capitalizing on the potential of data sharing, particularly during acute public health crises (Edelstein et al., 2018; Pisani et al., 2016). Yet, little research examines the practices and pitfalls of sharing public health data (Dye et al., 2016). Focusing on behavioral healthcare specifically, we were unable to find research that examined the need for sharing behavioral healthcare data at either a conceptual level or in terms of systems and practices for data sharing, despite the calls for data sharing in other health contexts. Knowledge Co-Production in Health Services Knowledge co-production is defined as the active engagement and involvement of actors in the production of knowledge, which takes place in processes that either emerge from or are facilitated and designed to accomplish such active involvement (Frantzeskaki & Kabisch, 2016). Knowledge co-production refers to knowledge created through interaction as well as to a culture that supports negotiating, creating, and sharing knowledge (Vindrola-Padros, 2018). There are two sets of conditions that influence the way knowledge is co-produced: (a) conditions surrounding the way knowledge co-production processes are set up like openness and inclusivity of the process; and (b) conditions related to the added value to society like usability of the knowledge co-produced (Frantzeskaki & Kabisch, 2016). Literature has examined various forms of co-production such as research co-production (Graham, 2019) and co-production of health services (Batalden et al., 2016). However, these concepts are distinct from knowledge co- production as we use it and as it is described in other disciplines. There is a growing body of literature regarding knowledge co-production in health services. In the health sector, knowledge co-production rests on the assumption that there are diverse stakeholders of healthcare services who all have valuable knowledge which is useful for The Journal of Community Informatics ISSN: 1721-4441 7 making system improvements (Vindrola-Padros, 2018). In contrast to the concept of knowledge translation, which focuses on synthesizing and accelerating innovations in knowledge (e.g. insights from health research) into practice (Greenhalgh & Wieringa, 2011), knowledge co- production focuses on producing knowledge collaboratively with relevant stakeholders. Because insights from knowledge co-production emerge directly from collaborative practice, knowledge co-production often leads to fewer difficulties in implementing new innovations because these innovations are rooted directly in the vested interests of stakeholders (Greenhalgh & Wieringa, 2011). There are calls from academics, policy analysts, and the popular press to share and combine data, thus creating integrated datasets that can be mobilized to illuminate and address major problems with health services delivery systems (Khoury & Ioannidis, 2014; Lohr, 2012; Raghupathi & Raghupathi, 2014). The literature on knowledge co-production suggests that initiatives that take a knowledge co-production approach may lead to higher levels of satisfaction among participants and more robust decision-making processes (Taylor-Phillips et. al, 2014). This led us to believe that developing a large multisector dataset about the behavioral health care delivery system would benefit from a knowledge co-production approach. However, while there is a growing body of literature on knowledge co-production in healthcare, we found no studies of knowledge co-production in healthcare focusing on knowledge co-production related to data sharing and analyses. Bridging Fragmentation through Boundary Organizations Boundary organizations are arrangements in which collaborators on either side of research, policy, or practitioner boundaries rely on an organizational arrangement to provide them with necessary resources (Guston, 2001), such as knowledge, data, and financial support. Boundary organizations involve participation by researchers and policymakers as well as professionals mediating between these groups. Further, boundary organizations provide incentives for the creation of boundary objects and are accountable to both research and policy communities (Cash, 2001; Crona & Parker, 2011; Guston, 2001; White et al., 2010). Thus, boundary organizations offer sites for collaboration, formation of new relationships, and the infusion of research and scientific information into policy (Schneider, 2009). Organizations from the local to the global scale have been identified and analyzed as examples of boundary organizations, such as Arizona State University’s Decision Center for a Desert City (Crona & Parker, 2011) and the Intergovernmental Panel on Climate Change (IPCC) (Humphreys, 2009; Hoppe & Wesselink, 2014). Boundary organizations are designed to bridge multisector fragmentation by 1) providing an opportunity and incentive for creation and use of standardized processes; 2) involving the participation of actors from both sides of the boundary and mediating professionals; and 3) existing at the border between multiple relatively different worlds with distinct lines of accountability to each (Guston, 2001). If successfully implemented, boundary organizations can develop less-politicized multisector collaborations where members of various stakeholder groups co-produce knowledge that will be utilized to bridge divergent worlds and inform decision making (Crona & Parker, 2012). They do this by providing a mechanism that both reinforces convergent The Journal of Community Informatics ISSN: 1721-4441 8 ideas between participants while allowing divergent ideas to continue (O’Mahony & Bechky, 2008). As previously mentioned, the U.S. healthcare system suffers from significant fragmentation across sectors. This fragmentation leads to significant “distance” between stakeholders which the literature suggests can make it more difficult to share data and communicate embedded knowledge utilized in practice (Carlile, 2002). Boundary organizations may present a useful means through which collaborators can organize around data sharing and interpretation. Our project was designed so that our research group and the university setting comprised a boundary organization that provided structure and a site of collaboration for multisector participants, representing almost a dozen organizations across five key service sectors. Methods Context This study occurred in Phoenix, Arizona. In Arizona, publicly funded behavioral health services are managed through organizations designated as regional behavioral health authorities (RBHAs), subcontractors responsible for managing service contracts, service capacity, and finances. In 2014, the RBHAs were required by the state Medicaid agency to integrate physical health care and behavioral health care for a certain class of patients who have been designated “serious mental illness” (SMI) patients. As a result of this data synthesis requirement, the data center at our university gained access to a large, state-wide dataset of Medicaid claims. This existing data holding made it much easier to activate a project geared toward integration of additional datasets that could shed light on issues regarding multisector services for patients with behavioral health illnesses. In addition, our project drew on the data visualization and convening facilities of the “decision theater” located within Arizona State University (hereafter referred to as DT). The DT is a unique center that partners with stakeholders and researchers to build computational models and convene diverse groups of decision makers across academia, government, and industry. The computational models integrate machine learning and predictive analytics to digest structured and unstructured data. We used a physical meeting space for the project referred to as the “Drum.” The Drum is a 26-foot-wide circular room that presents interactive models to participants across seven panoramic HD monitors in a 270°display. Project Our project followed Action Research (AR), defined as a systematic approach to research that focuses on developing solutions to problems encountered in everyday life (Stringer, 2013). Thus, AR provides a systematic and rigorous means for people to investigate problems and design means of addressing problems that is appropriate for their particular contexts and concerns (Stringer, 2013). We began by looking at the situation alongside community stakeholders. This largely occurred through past projects in which members of the research team and stakeholders The Journal of Community Informatics ISSN: 1721-4441 9 worked to address various problems with behavioral health services in Phoenix. These groups were hampered by the fact that many organizations serve the same populations, but their services, data, and efforts to solve problems are all siloed. To address this issue, the research team developed a novel intervention in collaboration with community stakeholders and applied for funding to support the intervention. Specifically, the intervention sought to develop a boundary organization that would convene multisector stakeholders and, using the existing resource of the university’s data center, integrate data from across the behavioral health system. Further, in addition to integrating datasets, the collective study team planned to hold multiple sessions in which community stakeholders would work with the data together to collectively make sense of system problems. Our intervention consisted of two parts starting in December 2017 and continuing over the next 1.5 years. The first part of the intervention consisted of the creation of an integrated multisector dataset through acquisition of patient-level datasets from five sectors related to behavioral health service delivery (behavioral healthcare, physical healthcare, jail, courts, and housing). The second part of the intervention consisted of design and interpretation of a series of data visualizations created using the newly acquired and integrated multisector dataset. To build our core team, we first needed to identify key community organizations in each of the relevant sectors for our target geographic area. We identified 50 organizations and invited them to send members to the kickoff meeting to introduce the project goals and timeline. 50 people attended the kickoff meeting. Of those who attended, 14 (described below) representing different organizations and all relevant sectors became highly engaged in the project, which required a substantial time commitment. Because participants in the upper levels of their organizations can influence the practices and policies of a given organization, most participants were in high-ranking roles with decision making authority. In future projects it may be valuable to enroll participants working at all levels of the care delivery system (including patients). The research team had prior knowledge of the functions and data capacities of these organizations as an outcome of past collaborative projects. The research team formed an advisory committee composed of members from key sectors to provide oversight and feedback on all aspects of the project. After the kickoff meeting, three workgroups with specific tasks were convened: 1) analytics and visualization, 2) dissemination and engagement, and 3) policy. The analytics & visualizations workgroup met three times to brainstorm the specific data analytics and visualizations they would like to carry out and design, assess patient-level data capacity for different organizations, and develop privacy safeguards for data. The dissemination and engagement workgroup met once to discuss successful engagement of participants over the course of the project. The policy workgroup met three times to discuss how the project could inform local, state, and national policy efforts. During this time, the research team worked with participants to secure permissions and access patient-level datasets from their respective sectors. The research team then secured and cleaned these datasets before integrating newly sourced datasets with existing data held in a data center at the university according to all applicable HIPAA and other patient and subject- safety regulatory requirements. By integrating newly accessed datasets with the current The Journal of Community Informatics ISSN: 1721-4441 10 Medicaid claims data and other holdings of the data center, we created a comprehensive integrated dataset that achieves interoperability at the level of the individual patient, over time, and across multiple related sectors. Integrated datasets included Medicaid claims, hospital discharge records, jail booking, and adult probation records. Data visualizations drawing on the integrated dataset were designed initially by the analytics and visualization working group and built by the DT programmer, and the research team plus the programmer presented the in-progress visualizations to participants during four data design meetings held in the Drum (Figures 2 and 3). Visualizations developed and refined through these meetings depicted multiple facets of interaction between different behavioral health sectors. Examples of visualizations include degree of integration between physical and behavioral health services for severe mental illness (SMI) and non-SMI patients and social network analysis of which services patients interact with across different data sets/sectors. Data design meetings were held roughly once a month over a four-month period. Each lasted two hours. The goal of these meetings was for participants to interact with the data visualizations and with each other, together driving the workshops through collaborative interpretations and making meaning of the data. Approximately 7-10 participants attended each of the data design meetings. Each meeting had at least one participant representing each of the five sectors engaged in the intervention. Privacy of individuals whose data were contained in the datasets was of utmost importance in the study design. To protect the privacy of individuals, aggregate, summary data were presented to participants outside the center. While we had capability to drill down to the individual level in data analysis the research team used differential privacy protections to protect the identity of individuals. Qualitative Evaluation of Knowledge Co-Production Process To understand the effect of the intervention, we conducted a qualitative evaluation. At each meeting multiple note takers took detailed notes to capture the ways participants discussed the data and visualizations, as well as the ways in which they contemplated real-world issues with behavioral health service delivery and how they connected the data visualizations to these service delivery problems. After each meeting the research team met to debrief and compare observations. In addition, the research team conducted semi-structured interviews with participants after the close of the project. The criteria for inclusion in the interviews was that the participant had to have attended both the kickoff meeting and at least three out of the four possible data design meetings. These criteria ensured feedback would be longitudinal, capable of reflecting on the ongoing process of collective data analysis and design of the data visualizations. From the 14 highly engaged participants who met this criteria, the research team interviewed 11 people. All interview participants served at a directorial position or higher within their organizations. Interviews lasted between 30 to 90 minutes and were audio recorded and transcribed. Observation notes and interview transcripts were analyzed using an interpretive approach (Tracy, 2013), characterized by careful study and close reading of recorded and The Journal of Community Informatics ISSN: 1721-4441 11 transcribed talk and text, using data exemplars and excerpts to support scholarly claims and arguments. The research team created a codebook and reread each of the interview transcripts, expanding on the beginning list of codes where necessary to create greater depth of understanding for emergent interview themes. After several meetings to discuss and compare the meaning, relevance, and salience of different codes, the research team reached consensus on a codebook for the final analysis. Data Driven Knowledge Co-Production Process The knowledge co-production process, which we refer to as the “Data-Driven Knowledge Co- production” or DDKCo process (Figure 1), involved four key practices centered on multisector data sharing and visualization which were carried out through “data design workshops.” Importantly, this process was nested within a boundary organization and required a crucial antecedent which we call protected time and space to interact. Boundary Organization The boundary organization consisted of three key elements: (1) organizational members; (2) a physical location; and (3) a specific organizing mechanism by which stakeholders interfaced with the research community. Figure 1: Process for Knowledge Co-Production in the Boundary Organization There were two primary categories of organizational members. The first included public stakeholders external to the university who contributed subject matter expertise regarding the provision of public and private sector behavioral health services, as well as the inter- The Journal of Community Informatics ISSN: 1721-4441 12 organizational relationships which constrained or enabled multisector service provision for people with serious mental illness. These included, but were not limited to, the following stakeholders: the Chief Medical Officer (CMO) for a state healthcare provider; a county-wide Director of Correctional Health Services; a crisis services liaison for a healthcare provider; Vice President of Clinical Services for a health organization; the CEO of a behavioral healthcare provider; the CEO of an organization with a focus on strengthening the Native American community; a Senior Director of Informatics for a healthcare provider; a police Detective/Crisis Intervention Coordinator; and the Deputy Chief of County Assessment and Programs for Probation. The second category included a cohort of university faculty with subject matter expertise in health information research, public health policy, capacity building in behavioral health services, health care finance, statistical programming, and biomedical informatics. Members of the faculty research team also brought methodological expertise and long-term working relationships with many of the organizations involved in the project, providing a grounded, contextual awareness of the local system. The physical location of the boundary organization was the DT Drum. Because of its community-engaged, solutions-focused heritage, convening organizational meetings within the physical location of the DT helped to support protected time and space for participants to engage with each other and the university. The primary organizing mechanism for the boundary organization was the repeated manipulation, analysis, visualization, and discussion of a large statewide data set compiled and organized by a center for health and information research located within the university, as part of an existing agreement with the state Medicaid agency. The data center has an ongoing series of projects to analyze and make use of the statewide Medicaid data set. By the time the boundary organization described here was formed, the data center had an established, credible reputation among physical and behavioral health care providers across the state. As a statewide bank of physical and behavioral health Medicaid claims, the existing dataset held by the data center was valuable but not useful. It was only through the convergence of multiple forms of expertise, active listening, and facilitated discussion that insights regarding potential uses and analysis of the data set emerged. Protected Time and Space to Interact A key element of the framework was protected time and space to interact with people from other sectors in the form of repeated meetings. Almost every participant commented on the role of these face-to-face interactions with diverse groups of participants from different sectors on their increased ability to understand and appreciate the complexity of approaches to behavioral healthcare. One participant, a Chief Medical Officer (CMO) for a healthcare provider described: “…the project created a reason for people to come together and talk…when a data slide was presented or when a hypothesis was provided, it allowed a reason for people who are otherwise very busy and doing their day-to-day work to say, ‘…I need to stop for an hour and look The Journal of Community Informatics ISSN: 1721-4441 13 and contemplate this with other stakeholders in the broader system.’ I think that creates interactions that don’t happen in real life [where] …everybody’s way too busy to think.” Thus, while data provided a justification to convene, our participants overwhelmingly reported that having protected time (in the eyes of their workplaces) and a space to gather together was one of the most important elements of our knowledge co-production process. Data Design Workshops The second key element were data design workshops, which contained four distinct and iterative knowledge co-production practices centered around the multisector datasets: 1) collective interaction with data and visualizations, 2) perspective taking, 3) reflection and debrief, and 4) iteration of visualizations. These practices tended to occur in the order listed, although these are not perfect “stages.” Next, we describe each practice. Collective Interactions with Data Interacting with data together helped multisector participants to grasp the complexity of the data collected and held by different sectors, as well as the disciplinary logics and other contextual factors that are embedded in datasets from different sectors. While participants had prior knowledge of the datasets collected and analyzed by their home organizations and sectors, for many participants the workshops were the first time they saw their data plotted next to another organization’s data and began to understand how other sectors depict their clients and services. When asked whether their participation in the project changed their understanding of the healthcare delivery system, a Police Detective and Crisis Intervention Training Coordinator recalled the first Data Design Meeting in the following excerpt: …we were just starting to look at preliminary data. A lot of the graphs were very physical, and mental claims data…looking at the rough data, side by side – and talking collaboratively about what conclusions we could draw from it sparked that discussion, and sort of made the biggest impact. Then, in successive meetings, as we were coming back to that topic and talking about it, it fleshes out what that actually means in practice. The CMO for a healthcare provider recalled the effect of collectively discussing “why this piece of data was important or this perspective important” to understand a given topic: It allowed people to say, ‘Well, what elements should be measured and included in the analysis?’ People…got into the challenges into how do you define that, how do you measure it, is it available, do we know it, all those kinds of things. Hearing what others thought was important to measure or to be able to evaluate was very helpful. Through collective interactions with data, participants also discussed social and ethical aspects of datasets. For example, during the Social Determinants of Health data design meeting, three participants (the Vice President of Clinical Services for a health organization, the CEO of a behavioral healthcare provider and the CEO of an organization with a focus on strengthening the Native American community) discussed which subgroups should be included for analysis. The VP commented that it might be hard to get substance abuse data to help understand trends in subgroups. The behavioral health CEO agreed that substance abuse data was important, saying The Journal of Community Informatics ISSN: 1721-4441 14 “…literature shows that, of people with SMI, 60 percent have substance abuse. But the way things get coded and paid for, those trends get masked.” The third participant quickly interjected that “the Native American population will not want substance abuse to be part of the determinant for SMI” and explained the issue from the perspective of her client population, including the ethical issues it could raise to examine substance abuse data alongside SMI data for her organization’s clientele. Thus, collective interactions with datasets and visualizations increased participants’ understanding of both the datasets held by other sectors and the actual services, cultures, and ethical concerns of other sectors. Perspective Taking We define the second practice, perspective taking, as the ability of diverse stakeholders to contextualize their positionality, experiences, and approach to behavioral health relative to the other participants in the workshops vis a vis datasets and visualizations. As they worked with data, participants let their expertise shine and often provided in-depth explanations and anecdotal examples to the group as they explained how data were collected, the limitations and implications of datasets, and how their home organizations used data. These interactions led participants to take the perspectives of participants in other sectors, seeing the behavioral health service system and client population from different vantage points beyond their own. Engaging with individuals during meetings led to seeing things from the point of view of participants from other sectors. The president of a behavioral healthcare agency described his experience in getting to know more about the perspective of a public health specialist, saying, There are things he thinks he understands but really doesn’t about health plans and the delivery system. By the same token, there are things I think I understand about what he’s doing, but I clearly don’t [laughs]. Part of the iterative process is coming to a level of understanding of each other’s worlds and what’s really needed as a whole to make it impactful. Participants reported a new understanding of the daily challenges that other participants face in carrying out their jobs related to behavioral health—for example, a behavioral health provider described how they gained a new appreciation for the intense “legal and regulatory barriers” that a police detective faced in doing their job. Examples such as this illustrate how our project allowed participants to understand the worldview of participants from other sectors. Participants also engaged in perspective taking on behalf of their agency and the sector that their agency represented. Throughout meetings, participants often used the collective “we” when referring to themselves, their coworkers, and their overarching organization and/or sector. This was reflected in statements such as “We engage in periodic follow-ups, we create intake forms for all patients and/or inmates, etc.” When asked if there was a time during the project when he was able to take the perspective of another agency, the Deputy CEO of a mental health services agency shared: … we’ve been able to take the perspective of [County] Correctional Health...They would describe bringing people in and what could’ve been successful alternatives to keep people in [behavioral health] care. They also talk about this ability to keep people out [of jail] by The Journal of Community Informatics ISSN: 1721-4441 15 making strong connections in the community. We started thinking through from their perspective what our outpatient teams could do a little bit better to support success and avoid recidivism. Another participant, a physician, described how a key undertaking of the workshops was gaining “the whole-system view,” which is crucial because “Everybody knew their trees or felt like they knew their own trees pretty darn well, but people really did want to see what others see and wanted to see the whole.” Reflection and Debrief Reflection and debrief occurred when participants connected data directly to service problems through reflecting on insights from the discussion of the data visualizations with diverse participants. For example, during the ‘high utilizers’ data design meeting, the group carried out a reflection activity, after which the system manager for a crisis services center described breakdowns in treatment and referral programs due to differences in transport reimbursements between psychiatric hospitals (reimbursed at $300) and medical hospitals (reimbursed at $1,200). They explained that patients with purely psychiatric needs would be transported to medical hospitals due to the higher reimbursement rate. There was an audible response from the entire workshop—the information was a complete surprise to most other participants and formed the basis of a conversation about misaligned incentives and the need for policy level change. Such reflections that connected data to real word problems often came at the very end of the workshop, once participants had time to center their thoughts, think about the data they had seen and discussion they had heard, and think critically through focused reflection activities about systemic breakdowns and opportunities for change moving forward. Dedicated time for reflection activities within the time constraints of the meetings offered time for participants to discuss the insights that they had together. Iteration of Visualizations During internal debriefs after each data design workshop, the research team shared their own key observations and insights after having listened to participants’ reactions to the visualizations and on-the-ground experiences. Through conversation between the research team and the visualization team, visualizations were then tweaked and re-designed. For example, one visualization focused on the utilization of different health services, including physical health, behavioral health, and pharmacy services, for the top five SMI- designated people who utilize health services overall. This visualization showed use of these different services on a time-lapse basis over the course of a year (Figure 2). It was not included in the first data design meeting. Rather, it was created based on participants’ comments of wanting to understand the highest utilizers as a starting point for intervention. After its creation and inclusion, participants found this visualization to be useful because it showed how individuals utilized different types of services, which allowed participants to discuss how patients interact with different sectors and the health care delivery system overall. After its first iteration, participants asked for more granular data about individuals who utilized different levels of The Journal of Community Informatics ISSN: 1721-4441 16 service. The internal team then iterated this visualization—the next version allowed participants to select a “decile” of utilization and see the time-lapse service utilization for the top five utilizers in each decile (figure 3). Figure 2: Time series representation of health utilization for the top 5 highest total cost SMI patients. Data has been slighted fuzzed to protect against re-identification. Gender has not been altered. Age (+/- up to 5 years). Cost amounts have been rounded ceiling/floor to nearest two thousand. Figure 3: A later version of the time series representation of health utilization for SMI patients. Like the first iteration of the visualization (Figure 2), this visualization shows time series data according to different categories of healthcare cost, and also allows the user to look at the top five utilizers along 10 deciles of healthcare utilization across the population. Iterating visualizations in turn led to a new round of collective interactions with data. Participants were then exposed to the updated visualization in later workshops. While the four knowledge co-production practices did not always happen in the chronological order reflected in Figure 1, and at times overlapped and melded together, the process outlined above is the best approximation of the temporal and practical ordering of the knowledge co-production practices that took place. The Journal of Community Informatics ISSN: 1721-4441 17 Evaluation: Outcomes of DDKCo Multisector Systems Awareness and Alignment of Systems Knowledge Because one of the primary goals of the project was to understand continuity of care across multiple health sectors, we asked participants about the extent to which they thought differently about problems in behavioral health systems after having participated in the project. Participants reported that they gained a new awareness of behavioral health systems. This awareness was specifically focused on a systems perspective that considered multiple sectors germane to behavioral health care. We refer to this type of awareness as multisector systems awareness. One awareness that participants reported was a new understanding of just how fractured data are across the multiple sectors and information systems that touch behavioral health patients. Participants also reported that they gained awareness of the breadth of the system. Meaning, their view of the “system” itself expanded beyond its former bounds, or they gained concrete evidence supporting their understandings of the expansiveness of the behavioral healthcare system. For example, a Senior Director of Informatics for a healthcare organization said: …[the project] provided more concrete thinking about the importance of these non- covered services…like housing and employment supports or things like that and just a better appreciation for how that can be thought of at a higher, broader level and any other results of those interactions or lack of those interactions. Finally, participants reported that they gained a sense of their own sector, and its attendant needs, priorities, and patterns of thinking, in relation to those of other sectors. A Deputy Chief of Assessment and Programs for Probation described: What I really noticed was that we all talk from our own discipline. I learned more about the health industry–I knew a lot about it–but I probably got more idea of what they pay attention to, their performance. What they’re looking at to improve their system...I definitely could look at [healthcare delivery] from a state Medicaid perspective or a health plan perspective, also probably the RBHA perspective. Engaging in perspective-taking during meetings led to a more durable ability to understand the broader behavioral health systems from the perspective of other sectors. Augmented Systems-Level Problem Solving Participants were aware of the limitations of prevailing approaches to decision making. For example, the CMO of a healthcare provider said: We are an organization that primarily uses…financial data. We use claims-based data…We intuitively know it’s important, but we do not use data from housing or justice The Journal of Community Informatics ISSN: 1721-4441 18 or other social sectors to be able to improve our insights, and it’s that kind of information that’s actually needed in order to be a little bit more proactive about the planning. Our interviews revealed that participants perceived that their participation in the DDKCo process had increased their ability to make decisions about behavioral health service delivery at the systems level. This increased ability to make systems-level decisions came from elevated awareness of the diverse consequences of their organization’s decisions on other organizations (and vice versa) and access to an integrated multisector dataset. Not only did participants acquire new abilities to engage in systems-level problem solving, but they also put these abilities to use using newly integrated datasets. This problem solving occurred through spontaneous creation of new projects by participants. For example, participants from an indigenous health organization, county corrections, and city government created a new project when they realized through the DDKCo process that the highest recidivism rates in the jail system were related to cases where individuals had experienced severe mental illness and homelessness. Participants created a new project to try and address the housing needs of this population to see if that would decrease rates of recidivism. Further, discussing services at the systems level pointed out goal misalignments between sectors. The discrepancies in reimbursements between psychiatric and medical hospitals raised by the crisis services center manager discussed earlier was a key example of misalignment between patient care and the health financial system. This example illustrates the potential for increased augmented decision making as a result of greater understanding of the effects of one organization’s decisions on another that arose from interpreting and reflecting upon data together. Finally, many meetings spawned discussions between participants about the usefulness of future data tools they would like to create. Throughout the interviews, participants echoed the realization that incorporating datasets from different sectors could help them improve or enhance the decisions they make regarding healthcare delivery for behavioral health patient populations. Prioritization of and Increased Knowledge about Data Sharing The final outcome revealed in our evaluation was prioritization of data sharing and increased knowledge about data sharing among participants. Before starting the project, many participants knew that sharing data between sectors might be beneficial, but they did not know which datasets existed nor how to go about sharing them. Participation in the project raised awareness of the need for data sharing, as well as the opportunities and limitations for data sharing present in their organizations and the broader context. For example, a former Senior Medical Director of a behavioral healthcare agency described the importance of integrating data: “The thing that would help would be to pool all of the data that we’re talking about that I believe is available to a degree and start sharing it better. To make it publicly available. To have datasets The Journal of Community Informatics ISSN: 1721-4441 19 as timely as we can. I understand not all data is in real time, but we can start analyzing and truly understand the need and understand the impact of our services.” This quote exemplifies a general sentiment expressed by participants that they were more aware, ready, and able to share data after participating in the DDKCo process. This is important since many of the participants are gatekeepers of data for their organization. Non-gatekeepers left the project with an increased sense of why data sharing is important and how it might be accomplished in practice. Discussion Through leveraging a boundary organization, the DDKCo process described above resulted in a newly integrated dataset that bridged fragmented behavioral health sectors. Equally important, our process produced a group of multisector participants who had new capacity to use the integrated dataset to address behavioral health problems fueled by knowledge co-production. This is evidenced by the outcomes uncovered in our qualitative evaluation: multisector systems awareness and alignment of systems knowledge, augmented systems-level problem solving, and increased knowledge about data sharing. As detailed in related work, approaches proposed to address data fragmentation in the literature thus far seek to reduce fragmentation through broad, high-level policy and large-scale technical interventions. In contrast, our intervention sought to address fragmentation through a local knowledge co-production process utilizing AR methods. Our approach resonates with literature on data sharing that shows that successful data sharing often draws on social relationships and an ability for sharers to draw on tacit knowledge about how data is collected and what they can make of it in their own practice (Faniel & Jacobsen, 2010; Wallis et al., 2013; Zimmerman, 2008). Critical data studies literature shows how all data are inevitably local, collected in particular times and places, for specific audiences that are conditioned to receive them (Loukissas, 2019). The university health data center had sufficient expertise to integrate datasets using patient metadata, clean, and curate data (although these activities of course required considerable work). What was most lacking was a social and institutional environment supportive of data sharing such that the needed datasets could be acquired with sufficient contextual information that they could be appropriately integrated, analyzed, and interpreted. Thus, our approach focused on creating the platform of human relationships needed for data sharing to occur and used data sharing and visualization as an occasion to build these relationships. Participants became institutional navigators who worked to gain needed permissions for data acquisition in their different sectors. Given how difficult data sharing of healthcare data has proven to be at a broad scale, we propose that our approach is a relatively expedient way to integrate datasets across sectors in specific geographic locales/service systems. Because the multisector system had neither established social relationships nor an understanding of the data held by different sectors, our intervention provided an opportunity for The Journal of Community Informatics ISSN: 1721-4441 20 participants to develop the necessary conditions (individual relationships and formal and tacit knowledge of one another’s work and data) to share data with one another. Through the DDKCo process, participants were able to create understanding and alignment in the meanings they attached to data (“data valences” in Neff & Fiore-gartland’s terms) (2015). To successfully share and analyze data across sectors, our findings point to the need to align data valences among these different sectors, and to the utility of knowledge co-production for doing so. Past community informatics literature has sought to increase use of open data initiatives and actionability of data by identifying appropriate data needed to support local actions, rather than publishing ‘disconnected data’ (Lupi et al., 2020). While not technically an “open data” project, our work supports Lupi et al.’s assertion that producing datasets in relation to particular needs is a fruitful approach, and further shows how knowledge co-production within a boundary organization is one potential way to do so. The goal of our process was not only for participants to share data across sectors, but to collectively interpret data as well. Visualizations, even scribbles on napkins during conversations, are powerful communicative devices (Snyder, 2014). Creating visualizations of the newly integrated dataset helped participants re-contextualize familiar data within a newly depicted “multisector system view.” Creating data visualizations together with other participants entailed discussion of what data analytics should be performed, how the population of patients should be “cut” for different data views, what questions were of most interest, and what privacy and ethical protections were needed. Carrying out these discussions with other participants over time as visualizations were designed, built, and tested allowed participants to gain an understanding of the “data frictions” (e.g. institutional and regulatory pressures) shaping their practice domains and datasets (Edwards et al., 2011; Nafus, 2014). Further, developing data visualizations helped participants to see their sector as part of a larger service system. The DDKCo process required participants to develop metaphorical “bifocal glasses” that allowed them to look closely at their own sector’s datasets, then take in the more distant, larger view of the multisector service system. This understanding, along with the relationships that were built, allowed participants to engage with the integrated data and visualizations to begin real-world problem solving, such as when participants began the spin off- projects to address homelessness as a factor in recidivism among the SMI population. Finally, our findings point to a new connection between the role of boundary organizations in initiatives to integrate data and co-produce knowledge among disparate groups, organizations, and sectors. Existing research supports the notion of employing boundary organizations as depoliticized arenas for work, which can lower cultural barriers between stakeholder groups and align their interests (Crona & Parker, 2011; O’Mahony & Bechky, 2008). Rather than isolating itself from external political authorities, our project’s boundary organization played an important role in negotiating knowledge between the science and policy realms (White et. al, 2010). It is important to note that this role does not have to be filled by a university. There are a number of non-academic organizations with the resources to support facilitated data sharing and design projects like the ones described here. This study inspires many opportunities to connect public health literature on multisector alignment with literature on data sharing, data visualization, and collaborative design. The The Journal of Community Informatics ISSN: 1721-4441 21 context for the design of information technology is increasingly an interconnected mosaic of responsive adaptive systems including people’s ideas, organizations, communities, markets, and culture (Furnas, 2000). Future studies that seek to support multisectoral, interdisciplinary, complex problems should attend to the relational component of these forms of work (Hinrichs, 2017). We argue that initiatives that seek to bridge sectoral divides must attend to relational data work (e.g. Meng et al., 2019; Møller et al., 2020; Pine & Bossen, 2020) that is necessary to successfully share and interpret data, and include resources to support these processes. Further, although the DDKCo process was informed in part by public health literature on multisector alignment, our approach could be used in any number of multisector problem spaces by developing appropriate boundary organizations and convening stakeholders to engage in data- driven knowledge coproduction. Lessons Learned A number of important lessons were learned over the course of the project that could inform subsequent projects. First, having a large dataset within the boundary organization (in this case the state Medicaid claims data) helped to facilitate data sharing because participants felt confident that procuring additional datasets would be beneficial due to the availability of existing data. It may be more difficult to acquire data from multiple organizations and sectors without forming a boundary organization that has data holdings already. Second, participants were largely working in high-level executive, managerial, or director roles in their organizations. As a result, knowledge co-production did not include knowledge from multisector front line workers and behavioral healthcare patients. It would be beneficial to include these perspectives in future versions of the project or similar projects. Third, it was challenging to iterate visualizations quickly. It would be beneficial to plan and create capacity for quick turnaround in data visualizations. Fourth, participants commented that it would be helpful for them to take visualizations back to their home organizations to engage colleagues in conversation about how the integrated datasets and data visualizations could be applied. It would be useful to create ways for non-co-located stakeholders to access data visualizations, and to explore ways that data design meetings could take place virtually or in addition to or in place of in situ meetings. Fifth, the DDDKCo process is resource intensive, requiring ongoing time and attention from participants. It is important to consider that stakeholders who are resource poor (for example, those who cannot allocate person hours to such an effort) may be excluded, leading to knowledge co-production that excludes some stakeholders. Future research should seek to understand the dynamics of inclusion and exclusion in the DDDKCo process and test mechanisms to ensure inclusive processes. Conclusion Sharing of healthcare data and mobilizing shared data for problem-solving is notoriously difficult due to fragmentation of healthcare providers and the fact that multiple sectors provide health- related services to the same populations. Large-scale data sharing and interoperability initiatives The Journal of Community Informatics ISSN: 1721-4441 22 have struggled. The process presented in this paper sought a new approach through using a boundary organization and knowledge co-production practices to collaboratively create an integrated multisector dataset and visualizations. This research extends literature on knowledge co-production by delineating a new process that can extend across disconnected and disparate social groups, helping different stakeholder groups make meaning of data related to overlapping populations, align datasets and problem-solving efforts from the local organizational level to the multisector system level, and lay the relational groundwork for knowledge sharing across siloed health providers. This work also highlights the important role that boundary organizations can play in facilitating data sharing and alignment of multiple health sectors. Acknowledgements We thank the participants and partner organizations in this project without whom this work would not have been possible. We also wish to thank the editor and reviewers for their thoughtful reviews which helped us improve the paper. This work was funded by a grant from the Robert Wood Johnson Foundation (no. 75584). Arizona State University is located in the Salt River Valley on ancestral territories of twenty-two Native nations who have inhabited this land for centuries, including the Akimel O’odham (Pima) and Pee Posh (Maricopa) Indian Communities. We acknowledge the sovereignty of these peoples, whose care and keeping of these lands allows us to be here today. References Ackoff, R.L. (1998). A systemic view of transformational leadership. Systemic Practice and Action Research, 11,1, 23– 36. Alvarado Garcia, A., Young, A. L., & Dombrowski, L. (2017). On making data actionable: How activists use imperfect data to foster social change for human rights violations in Mexico. Proceedings of the ACM on Human-Computer Interaction, 1(CSCW), 1-19. Batalden, M., Batalden, P., Margolis, P., Seid, M., Armstrong, G., Opipari-Arrigan, L., & Hartung, H. (2016). Coproduction of healthcare service. BMJ quality & safety, 25(7), 509-517. Bhugra, D., Kar, A., and Lawton-Smith, S. (2014). Integration of mental and physical health services: lessons. J. Psychosoc. Rehabil. Ment. Health 1, 15–21. DOI: https://doi.org/10.1007/s40737-014-0004-3 Carlile, P.R. (2002). A pragmatic view of knowledge and boundaries: Boundary objects in new product development. Organization science, 13(4), 442-455. Cash, D.W. (2001). “In order to aid in diffusing useful and practical information”: agricultural extension and boundary organizations. Science, Technology, and Human Values, 26(4), 431-453. The Journal of Community Informatics ISSN: 1721-4441 23 Cragin, M. H., Palmer, C. L., Carlson, J. R., & Witt, M. (2010). Data sharing, small science and institutional repositories. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 368(1926), 4023-4038. Crona, B.I. & Parker, J.N. (2011). Network determinants of knowledge utilization: preliminary lessons from a boundary organization. Science Communication, 33(4), 448-471. Crona, B. I., & Parker, J. N. (2012). Learning in support of governance: theories, methods, and a framework to assess how bridging organizations contribute to adaptive resource governance. Ecology and Society, 17(1). Dye, C., Bartolomeos, K., Moorthy, V., & Kieny, M. P. (2016). Data sharing in public health emergencies: a call to researchers. Bulletin of the World Health Organization, 94(3), 158-158. Edelstein, M., Lee, L. M., Herten-Crabb, A., Heymann, D. L., & Harper, D. R. (2018). Strengthening global public health surveillance through data and benefit sharing. Emerging Infectious Diseases, 24(7), 1324. Edwards, P. N., Mayernik, M. S., Batcheller, A. L., Bowker, G. C., & Borgman, C. L. (2011). Science friction: Data, metadata, and collaboration. Social studies of science, 41(5), 667-690. Faniel, I. M., & Jacobsen, T. E. (2010). Reusing scientific data: How earthquake engineering researchers assess the reusability of colleagues’ data. Computer Supported Cooperative Work (CSCW), 19(3), 355-375. Fiore-Gartland, B. & Neff, G. (2015). Communication, mediation, and the expectations of data: Data valences across health and wellness communities. International Journal of Communication, 9(19), 1466-1484. Frank, M., Walker, J., Attard, J., & Tygel, A. (2016). Data Literacy-What is it and how can we make it happen?. The Journal of Community Informatics, 12(3). Frank, M., & Walker, J. (2016). Some key challenges for data literacy. The Journal of Community Informatics, 12(3). Frantzeskaki, N. & Kabisch, N. (2016). Designing a knowledge co-production operating space for urban environmental governance—lessons from Rotterdam, Netherlands and Berlin, Germany. Environmental Science & Policy, 62, 90–98. doi: 10.1016/j.envsci.2016.01.010 Furnas, G.W. (2000). Future design mindful of the MoRAS. Human-Computer Interaction, 15(2), 205-261. Gay, V., & Leijdekkers, P. (2015). Bringing health and fitness data together for connected health care: mobile apps as enablers of interoperability. Journal of medical Internet research, 17(11), e260. Graham, I.D., McCutcheon, C., & Kothari, A. (2019). Exploring the frontiers of research co- production: the Integrated Knowledge Translation Research Network concept papers. Health Res Policy Sys 17(8), 1-4. https://doi.org/10.1186/s12961-019-0501-7 The Journal of Community Informatics ISSN: 1721-4441 24 Greenhalgh, T. & Wieringa, S. (2011). Is it time to drop the 'knowledge translation' metaphor? a critical literature review. J R Soc Med. 104(12), 501–509. doi:10.1258/jrsm.2011.110285 Guston, D.H. (2001). Boundary organizations in environmental policy and science: an introduction. Science, Technology, Human Values, 26(4), 399-408. Hinrichs, M. M., Seager, T. P., Tracy, S. J., & Hannah, M. A. (2017). Innovation in the Knowledge Age: implications for collaborative science. Environment Systems and Decisions, 37(2), 144-155. Hoppe, R. & Wesselink, A. (2014). Comparing the role of boundary organizations in the governance of climate change in three EU member states. Environmental science & policy, 44, 73-85. Howarth, C., & Monasterolo, I. (2017). Opportunities for knowledge co-production across the energy-food-water nexus: Making interdisciplinary approaches work for better climate decision making. Environmental Science & Policy, 75, 103-110. Humphreys, D. (2009). Working across boundaries: science–policy interfaces and international forest politics. Environmental Sciences, 6(3), 163-174. Kasthurirathne, S. N., Mamlin, B., Kumara, H., Grieve, G., & Biondich, P. (2015). Enabling better interoperability for healthcare: lessons in developing a standards based application programing interface for electronic medical record systems. Journal of medical systems, 39(11), 1-8. Khoury, M.J. & Ioannidis, J.P.A. (2014). Big data meets public health. Science, 346(6213), 1054- 1055. Liyanage, H., Krause, P., & De Lusignan, S. (2015). Using Ontologies to Improve Semantic Interoperability in Health Data. Journal of innovation in health informatics, 22(2), 309-315. Lohr, S. (2012). The age of big data. New York Times (November 2012). Loukissas, Y.A. (2019). All Data are Local: Thinking Critically in a Data-driven Society. Mit Press. Lupi, L., Alessio, A., De Liddo, A., & Motta, E. (2020). Actionable Open Data: Connecting City Data to Local Actions. Journal of Community Informatics, 16, 3-25. Marceglia, S., Fontelo, P., & Ackerman, M. J. (2015). Transforming consumer health informatics: connecting CHI applications to the health-IT ecosystem. Journal of the American Medical Informatics Association: JAMIA, 22(e1), e210. Meadow, A. M., Ferguson, D. B., Guido, Z., Horangic, A., Owen, G., & Wall, T. (2015). Moving toward the deliberate coproduction of climate science knowledge. Weather, Climate, and Society, 7(2), 179-191. Meng, A., DiSalvo, C., & Zegura, E. (2019). Collaborative data work towards a caring democracy. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), 1-23. Møller, N. H., Bossen, C., Pine, K. H., Nielsen, T. R., & Neff, G. (2020). Who does the work of data?. Interactions, 27(3), 52-55. The Journal of Community Informatics ISSN: 1721-4441 25 Nafus, D. (2014). Stuck data, dead data, and disloyal data: The stops and starts in making numbers into social practices. Distinktion: Scandinavian Journal of Social Theory, 15,2, 208-222. Namatovu, E., Sæbø, J. I., & Kaasbøll, J. J. (2017). Mobilities of the Community Health Work Practice: Mobile Health System Mediated Work. The Journal of Community Informatics, 13(3). National Alliance on Mental Illness. (2019). Mental Health By the Numbers. Retrieved from https://nami.org/mhstats. Obeysekare, E. R., Mehta, K., & Maitland, C. (2017). Bringing Community Back to Community Health Worker Studies: Community interactions, data collection, and health information flows. The Journal of Community Informatics, 13(3). O'Mahony, S. & Bechky, B.A. (2008). Boundary organizations: enabling collaboration among unexpected allies. Administrative science quarterly, 53(3), 422-459. Ottinger, G. & Zurer, R. (2011). Drowning in data. Issues in Science and Technology, 27(3), 71-82. Pine, K. H., & Bossen, C. (2020). Good organizational reasons for better medical records: The data work of clinical documentation integrity specialists. Big Data & Society, 7(2), 2053951720965616. Pisani, E., Aaby, P., Breugelmans, J. G., Carr, D., Groves, T., Helinski, M., ... & Guerin, P. J. (2016). Beyond open data: realising the health benefits of sharing data. BMJ (Clinical research ed.), 355, i5295. Raghupathi, W. & Raghupathi, V. (2014). Big data analytics in healthcare: promise and potential. Health information science and systems, 2(1), 3. Rowhani-Farid, A., Allen, M., & Barnett, A. G. (2017). What incentives increase data sharing in health and medical research? A systematic review. Research integrity and peer review, 2(1), 1- 10. Rycroft-Malone, J., Burton, C. R., Bucknall, T., Graham, I. D., Hutchinson, A. M., & Stacey, D. (2016). Collaboration and co-production of knowledge in healthcare: opportunities and challenges. International journal of health policy and management, 5(4), 221. Schneider, A.L. (2009). Why do some boundary organizations result in new ideas and practices and others only meet resistance? Examples from juvenile justice. The American Review of Public Administration, 39(1), 60-79. Snyder, J. (2014). Visual representation of information as communicative practice. Journal of the Association for Information Science and Technology, 65(11), 2233-2247. Stringer, E. T., & Aragón, A. O. (2020). Action research. Sage publications. Taylor-Phillips, S., Clarke, A., Grove, A., Swan, J., Parsons, H., Gkeredakis, E., ... & Scarbrough, H. (2014). Coproduction in commissioning decisions: is there an association with decision satisfaction for commissioners working in the NHS? A cross-sectional survey 2010/2011. BMJ open, 4(6). The Journal of Community Informatics ISSN: 1721-4441 26 Tracy, S.J. (2019). Qualitative research methods: Collecting evidence, crafting analysis, communicating impact. John Wiley & Sons. Vindrola-Padros, C., Eyre, L., Baxter, H., Cramer, H., George, B., Wye, L., ... & Marshall, M. (2019). Addressing the challenges of knowledge co-production in quality improvement: learning from the implementation of the researcher-in-residence model. BMJ quality & safety, 28(1), 67-73. Wallis, J. C., Rolando, E., & Borgman, C. L. (2013). If we share data, will anyone use them? Data sharing and reuse in the long tail of science and technology. PloS one, 8(7), e67332. White, D. D., Wutich, A., Larson, K. L., Gober, P., Lant, T., & Senneville, C. (2010). Credibility, salience, and legitimacy of boundary objects: water managers' assessment of a simulation model in an immersive decision theater. Science and Public Policy, 37(3), 219-232. Zimmerman, A.S. (2008). New knowledge from old data: The role of standards in the sharing and reuse of ecological data. Science, Technology, & Human Values, 33, 5, 631-665.