1425 final The Journal of Community Informatics ISSN: 1721-4441 Articles Using Q-Sort Methodology to test the Non- hierarchical Online Learning Community (NHOLC) Framework The Non-Hierarchical Online Learning Community (NHOLC) conceptual framework was designed to leverage the understanding of sociocultural learning theory and community informatics to inform design principles for citizen science online learning communities that inspire online collaboration and local environmental action. The study presented here applies the NHOLC framework, using a Q-Sort methodology, to three online learning communities for citizens that were successful in fostering online collaboration and environmental actions. The findings of this paper provide tangible design principles that can be used to develop or revise online learning communities for citizen science instead of re- inventing the wheel for each newly emerging project. Ruth Kermish-Allen Maine Mathematics and Science Alliance, USA rkermishallen@mmsa.org Corresponding author Kate Kastelein Maine Mathematics and Science Alliance, USA kkastelein@mmsa.org !17 Kermish-Allen, R., Kastelein, K. (2018). Using Q-Sort Methodology to test the Non-hierarchical Online Learning Community (NHOLC) Framework. The Journal of Community Informatics, 14(2), 17–-38. Date submitted: 2017-11-02. Date accepted: 2018-11-22. 
 Copyright (C), 2018 (the author as stated). Licensed under the Creative Commons Attribution- NonCommercial-ShareAlike 2.5. Available at: www.ci-journal.net/index.php/ciej/article/view/1425 . mailto:rkermishallen@mmsa.org mailto:kkastelein@mmsa.org http://www.ci-journal.net/index.php/ciej/article/view/1425 http://www.ci-journal.net/index.php/ciej/article/view/1425 The Journal of Community Informatics ISSN: 1721-4441 Introduction Citizen science projects have traditionally been based on the scientific community’s need to gather vast quantities of high quality data, neglecting to ask what the project participants get in return. How can participants be seen more as collaborative partners in citizen science projects? Online communities for citizen science are expanding rapidly, giving participants the opportunity to take part in a wide range of activities, from monitoring invasive species to identifying far-off galaxies. These communities can bring together the virtual and physical worlds in new ways that are egalitarian, collaborative, applied, localized and globalized to solve real environmental problems. This vision is in direct alignment with that of the community informatics (CI) field. For example, CI is defined as the use of data from various technological communications from within a community (Dara O’Neil, 2002), with the goal to use the data to improve the community (Stoecker, 2005). Application of CI to improve the community can relate to community sustainability, learning opportunities, workforce development (Eagle, Hague, Keeble, & Loader, 2005), and can facilitate increased civic engagement (Stoecker, 2005). With this in mind, CI and the more participatory and co-created variants of citizen science have a great deal to learn from each other. The study presented here is designed to dive deep into citizen science projects that are realizing this shared vision of citizen science and CI to provide insights into how the field can learn from examples of successes and amplify outcomes in online collaboration and solutions to real environmental issues that matter to participants. Defining the NHOLC Conceptual Framework The Non-hierarchical Online Learning Community (NHOLC) conceptual framework (Kermish-Allen & Kastelein, in press) provides a powerful starting point for designing and studying online spaces. The NHOLC framework is rooted strongly the sociocultural learning theories of Communities of Practice (Lave & Wenger, 1991; Wenger, 2000b, 2000a), Knowledge Building (Scardamalia & Bereiter, 2006), Place-Based Education (Smith & Sobel, 2010; Sobel, 2005), and Funds of Knowledge (Gonzalez, Moll, & Amanti, 2005; Moll, Amanti, Neff, & Gonzalez, 1992). Communities of Practice (CoP), focuses on how a group of individuals work and learn together. Integration of CoP theory helps inform how an online community might function. Place-Based Education (PBE) links the work of the community to the relevant interests and place of the participants. Knowledge Building (KB) guides the community with the intention to build new knowledge together related to the citizen science question mind. Funds of Knowledge (FoK) provide the framework with guidance for how to value diverse lived experiences and not just the “usually suspected” forms of expertise. !18 The Journal of Community Informatics ISSN: 1721-4441 This conceptual framework incorporates diverse participant groups, real-world investigations rooted in place (local contexts), valuing lived experience as essential to building new knowledge, a recognition that knowledge generation is not a top-down process but instead a dynamic multi-directional process between participants, and finally leveraging the power of a digital culture to build a knowledge-building community that transcends geographic limitations of traditional place-based education to answer questions people care about. Strengths of the NHOLC Framework Image 1: Visual representation of the theoretical frame of NHOLC conceptual framework This interwoven framework places emphasis on: 1. Bringing together diverse participant groups from widely differing areas of expertise to enable multi-directional learning opportunities in which everyone who joins the community has something to offer and teach others within the community !19 The Journal of Community Informatics ISSN: 1721-4441 2. Enabling participant-driven real-world investigations personally relevant to participants’ lives 3. Sharing project purpose and goals 4. Enabling communication structures to build relationships and roles among diverse participants 5. Sharing place-based data across geographic boundaries Image 1 above highlights how the core concepts of each guiding theory have been woven together into the five focus areas of the NHOLC conceptual framework. As the strengths of these theories weave together to create a new genre for collaborative online learning – the NHOLC framework – the question we now must ask is: which components of this new framework are most important for fostering collaboration in online learning communities? Understanding which components are most important to online collaboration can begin to define design principles for successful applications of the NHOLC framework. As described at the start of this paper the realm of citizen science provides a valuable context for the use of the NHOLC framework. To test this proposition, the sections below detail a Q Methodology study that addressed the research question: what are the essential theory-based design elements of online learning communities for citizen science projects that resulted in environmental actions? Methods – Identifying the Cases To answer this question the most appropriate strategy is to find best-case scenario examples of projects that have characteristics of the NHOLC framework embedded in their design. With this in mind, a multiple-case study design is the best methodological fit to understand the implementation and design of an innovation (Yin, 2014), in this case online learning communities for citizen science that have been successful in supporting online collaboration and on-the-ground environmental actions. A wide variety of citizen science projects use online learning communities. However not all are locally focused or personally relevant for the individual participant. For example, Galaxy Zoo asks members to classify distant galaxies from images captured by the Hubble telescope and the Sloan Digital Sky Survey. Others, such as the 1000 Genome project, asks for help from the general public to identify novel genetic variants in examples of Y chromosomes from across Europe to track the historical migrations of humans. These are exciting ways to engage in citizen science and scientific questions but they do not necessarily relate to a participant’s everyday life and the world around them. The filter this study chose to use was projects that have shown success in implementing local environmental actions due to engaging in online collaboration. “Action” in this sense does not refer to modifying specific individual behaviors like recycling or saving water, but instead to engage learners in planning and taking action on !20 The Journal of Community Informatics ISSN: 1721-4441 community-level environmental issues they find relevant (Schusler, Krasny, Peters, & Decker, 2009). Results of these types of action can be seen in policy changes such as land use regulations to conserve or restore sensitive habitats; efforts to eradicate invasive species; sociological changes that promote car pooling or public transportation use; and activities such as planting butterfly gardens. Participants learn to critically analyze information, make informed decisions, and take an active role in accomplishing tasks to enable those actions. The key components of the action of interest in this study are solutions that the citizen scientists produced to address the problems they identified. This study does not analyze the actions in any way: it simply documents the types of actions resulting from the projects and the design elements and practices present in the project that supported its development. The site selection requirements can be found in Box A (below). To tease out the most important, or essential, components of these online communities a Q-methodology is used to assess participants’ priorities about an issue. The issue in this case is – what are the most important components of the NHOLC framework that foster collaboration and environmental action in online learning communities for citizen science. Box A: Site selection criteria !21 Site Selection Criteria 1. The citizen science project uses an online space to: a. Bring multiple stakeholders together to answer questions relevant to the project (diverse ages and areas of expertise – for example, teachers, students, scientists, interested citizens, etc.) b. Bring together geographically diverse individuals to share place-based data c. Upload and share data via mobile and/or desktop technology with all users d. Analyze data e. Identify new areas of inquiry f. Provide an opportunity for users to connect with each other via a variety of means (discussion posts, messages, etc.) 2. The online space has: a. Been in use for 6 months or more b. Has funding to remain active through June 2016 c. The ability to record and capture discussions online between users 3. The overall citizen science project: a. Have evidence of environmental actions implemented by participants due in part to the project b. Be committed to working with individuals of different ages and expertise (e.g., youth and adults; students; civic and science professionals) c. Be willing to share activity log data with this researcher d. Be willing to reach out to project participants for surveying and interviewing purposes e. Be willing to support staff time and opportunities to participate in researcher interviews and observations The Journal of Community Informatics ISSN: 1721-4441 Q – Methodology Q-methodology, originally developed by William Stephenson (1935) to assess individuals’ priorities about an issue, is designed to recognize the different value systems of different constituents (Brown, 1980). This approach can also illustrate underlying patterns between groups or individuals that have broad shared values, and can capture ‘the way in which meaning is organized and patterned’ (Brewerton & Millward, 2001). The basic difference between Q-methodology and standard survey analysis is its design to establish patterns within and across individuals rather than patterns across individual traits, such as age, class, etc. (Barry & Proops, 1999). In this study, Q-methodology uncovers the most important components of the NHOLC framework – the underlying essential design elements for collaboration and local environmental action common across individuals’ experiences of online learning communities for citizen science. A total of 30 citizen science participants took part in the study. The sample includes project coordinators/founders and other participants including scientists, teachers, local community organizers, and the general public. These individuals came from across the country including Maine, California, New York City, New Orleans, and everywhere in between. All participants are active on one of the online communities. The sample includes all the major types of stakeholders involved in each of the case’s learning communities. Fifteen participants, five from each of the three cases comprise this Q-sort. Each case includes one project founder/coordinator, two scientists or technical experts, and two members of the general public at varying levels of participation. Descriptions for Case Study Sites Public Lab is both an online community and a non-profit that grew out of a grassroots initiative during the Deepwater Horizon BP Oil Spill to enable communities impacted by the spill to access data from “community satellites” - helium balloons, kites with mounted inexpensive digital cameras - about where and how much the oil was spreading. Public Lab participants can learn how to investigate environmental concerns of interest to them using inexpensive Do-It-Yourself (DIY) techniques. Today, Public Lab is an international community with participants active in every hemisphere. The types of environmental monitoring range from water quality to air pollution, and others as the community defines its interests. The online community is an open network of educators, technologists, scientists, and researchers working to create, share, and use low cost solutions to solving local environmental problems. This community has supported environmental actions ranging from aerial mapping for monitoring purposes (measuring waterway pollutants, drought conditions, plant health, invasive species, industrial pollution, etc.) to water quality testing to air quality monitoring—with opportunities to engage in local actions related to the research. !22 The Journal of Community Informatics ISSN: 1721-4441 The goal of Vital Signs, a project of the Gulf of Maine Research Institute, is identifying and documenting invasive plants in the Northeast United States. The project originated as a citizen science project focused for the K-12 classroom, but has since grown to include adults at environmental organizations like land trusts, master gardeners, and others. The online community provides a venue to learn about which species others are finding in the region, participate in “missions” to find specific invasive species, and then a space to upload their findings. Participating scientists, or species experts, confirm or deny species identification of user data. Recently, Vital Signs has added a “design your own mission” function that allows participants to design their own investigations into local environmental questions. Many participants have engaged in action by hosting community events to educate the public about the presence, spread, and concerns connected with specific invasive species. Others have conducted removal or remediation of invasive species. WeatherBlur, a project of the Maine Math and Science Alliance, brings together scientists, fishermen, and K-8 students and teachers to explore the local impacts of shifting weather and climate change via an online community. The WB learning community uses online technologies to provide users with the opportunity to participate in an evolving set of “cocreated” citizen science projects. The projects are rooted in place-based weather and climate data and questions that matter to citizens and provide highly valued data to scientists. During the pilot phase of this project, a bycatch study used lobster traps to investigate organisms that live among lobsters. Members asked questions about each other’s data and provided suggestions for more accurate data collection. In August 2014, a task force on green crabs solicited a summary of the WeatherBlur investigation. This report played an important role in developing new regulations to minimize threats posed by the crabs. Each of the cases described above provide an online space for diverse stakeholders - scientists, youth, teachers, interested community members such as representatives from natural resource based economies - to ask questions, share and analyze data, collaboratively solve personally relevant scientific questions, and build new knowledge together. The goal behind all these projects is to increase the participatory involvement of local people in environmental monitoring that can lead to highly accelerated research findings and policy changes to tackle environmental challenges. Development of Q- Statements Q-methodology typically includes a concourse stage during which statements are generated to capture the full range of subjective experiences on the study topic. The most representative statements constitute the Q-sample (McKeown & Thomas, 2013). The Q- statements for this study were designed to highlight two different contexts. First, a series of 45 statements reflect the core ideas of the original four socio-cultural learning theories - community of practice, knowledge building, place-based education, and funds of knowledge – gathered largely from existing literature and design principles specific to each !23 The Journal of Community Informatics ISSN: 1721-4441 individual learning theory. Second, a series of 55 statements reflect the emerging ideas of interwoven NHOLC conceptual framework. Each set of statements were then culled to limit repetition, increase clarity, and to reach a manageable number of unique statements that reflected the range of literatures reviewed. In accordance with the recommendations of McKeown & Thomas (2013), the final set of statements were limited to no more than 50 statements. The final 49 statements included in the Q sample are presented in Table 1 in the findings section. The first column of the table denotes the origination of each statement – original theoretical frame or NHOLC conceptual frame. Any statement with a NHOLC represents the new interwoven framework. Data Collection The Q-sort template for this study was forced-choice and arranged in a quasi-normal pattern (Figure 2). Statements were sorted on a nine-point scale, ranging from -4 (Least Important) to +4 (Most Important), as shown in Figure 1. Each of the three case study projects – WeatherBlur, Public Lab, and Vital Signs – were all equally represented with 10 participants each. To ensure equal representation only adult participants from the United States were included in this study. In addition, the different participant groups from each project were equally represented in the sample of 10 individuals from each project including research scientists/technical experts, project coordinators, teachers, local activists, and general participants. Q-sorts were completed using online video conferencing and screen sharing. Each interview was approximately 60 minutes. The remote participants completed the Q-sort online using Flash-Q software (Hackert & Braehler, 2007). The interview began with an introduction to how the online flash version Q sort functioned. The participant manually sorted each statement with his or her own mouse. All 30 participants were presented with the statements in random order, and asked to arrange them initially into one of three piles (important, neutral, not important). Next, participants were asked to sort their cards according to the template below by placing cards in one of the nine columns – forced choice placement (see Figure 2). After completing the sort, participants were interviewed to document the reasons they selected statements as most or least important and to gather their perceptions of the overall themes for the kinds of statements that were most and least important to productive collaboration in an online citizen science community. Each Q-sort and associated interview took no more than 75 minutes per person total and was completed in one session. !24 Prompt Used During Q-sorts with Participants “We are interested in learning about the pieces of the _______ project that were most successful at helping participants work together on the project to meet goals. Please think about the _______ project and sort the statements below to identify those that are most and least important for collaboration online with other participants to reach the goals of the project.” The Journal of Community Informatics ISSN: 1721-4441 Image 2: Q-sort Distribution for 49 Q Statements Defining Factors: Perspectives on what was most important for online collaboration and on-the-ground action The Q-sort data were analyzed using PQMethod software (Schmolck, 2002), which conducts a Principle Components (Factor) Analysis with a Varimax rotation. Following the criteria set by Green and Salkind (Green & Salkind, 2010), the three-factor solution was chosen to interpret the range of perspectives captured. The three-factor solution with the equitable representation across projects accounts for all individuals in the sample, and explains 49% of the total variance. Table 1 (below) shares the findings of the Q sort. Column 1, theoretical frame, denotes whether the statement originated from one of the four original theories alone (CoP, KB, FoK, or PBE) or if it is a statement describing the new interwoven NHOLC conceptual framework (XXX/NHOLC). Column 2, statement, simply restates each of the statements that participants ranked during the Q sort. Columns 3, 4, and 5 Factor #, provides the average ranking for each statement from the individuals that were grouped together into each factor. Column 6 simply restates which factor each statement fit best within based on the amount of agreement amongst the individuals in that factor. For example, the first 11 statements in the table are statements that all members of factor 1 agree with. The statements that have a “1,2,3” in this column are statements that could fit into each of the separate factors, but each factor had a different perspective on how important or unimportant it was. The statements that have an “NA” in this column are statements that did not fit into any factor. The statements that have a “consensus” in column 6 are statements that everyone in the entire sample shared agreement on. !25 The Journal of Community Informatics ISSN: 1721-4441 See Table 1 for the factor rankings of Q-sort statements, including consensus rankings. Table 1: Factor ranking of Q-sort statements and consensus statements Theoretical frame Statement Factor 1 Factor 2 Factor 3 Three Factor Solution KB The online learning community provided a structure to encourage the sharing of responsibilities and decision-making. -3 0 -1 1 CoP/NHOLC The online learning community connected individuals who have similar interests, but did not use the same resources for work (the same language, tools, experiences, definitions). -2 0 0 1 CoP/NHOLC The different perspectives of online learning community members assisted in developing individuals’ roles on the online learning community. -3 -1 -1 1 CoP/NHOLC The online learning community encouraged members to value the variety of expertise present in the community. -1 1 1 1 CoP/NHOLC The different perspectives of online learning community members aided in developing relationships with others in the community. -2 0 1 1 FoK/NHOLC The online learning community encouraged members with historic and cultural knowledge relevant to the project to share that knowledge with others. -2 1 0 1 KB/NHOLC The online learning community brought together the diverse stakeholders needed to achieve the project’s goals. 0 2 2 1 CoP The online learning community’s overall shared purpose motivated members of the community. 3 -2 -1 1 KB online learning community members had a commitment to the same overall goals. 3 -3 -2 1 KB/NHOLC The online learning community had a mechanism that provided the opportunity to critique and help shape new ideas that emerge from the members of the community. 1 -3 -2 1 KB online learning community members had a commitment to building knowledge that could be used by the whole community. 1 -1 0 1 !26 The Journal of Community Informatics ISSN: 1721-4441 CoP/NHOLC The online learning community attracted new members by showing the relevance of the project to potential member’s lives and interests. -1 1 -2 2 CoP The online learning community provided a starting point for conversation. 1 3 1 2 KB/NHOLC The online learning community encouraged any community member (no matter his/her age, expertise, or perspective) to propose new questions or investigations on the site. 0 2 0 2 KB/NHOLC The online learning community provided the opportunity for members from multiple perspectives to respond to and build on the ideas of others to advance a project. 1 4 0 2 CoP The online learning community’s overall shared purpose united members of the community. 0 -4 0 2 CoP Online learning community members felt like they were working toward the common goal of building new knowledge together. 2 -4 2 2 KB/NHOLC Members joined the online learning community because they wanted to build knowledge related to the shared goals of the project. 2 -3 2 2 CoP/NHOLC A shared purpose was important in fostering collaboration on the online learning community amongst its various stakeholders. 1 -1 1 2 CoP online learning community members had the ability to move from new-comer to experienced members as they enhanced their skills and relationships on the site. -1 0 3 3 CoP The online learning community provided members with the freedom to express opinions and offer suggestions without fear of how the other members would judge it. -1 -2 2 3 KB The online learning community provided all members with a way to track and understand how and why a project changed over time. 0 0 -4 3 KB The online learning community provided the opportunity to develop investigations that represented evolving ideas in the community. 1 0 -1 NA CoP/NHOLC The OLC provided a starting point for discussion with stakeholder groups that otherwise not connect to share ideas. 0 2 1 NA !27 The Journal of Community Informatics ISSN: 1721-4441 CoP The online learning community provided the opportunity for community members to develop roles on the site. -4 -2 -4 NA CoP/NHOLC The online learning community brought together people with different levels of expertise and/or experience. 1 2 3 NA KB/NHOLC All members of the online learning community had the potential to influence the direction and focus of projects. 0 3 -1 1, 2, 3 CoP The online learning community provided a place to put resources that were used by the community. 3 1 -1 1, 2, 3 CoP The online learning community had a structure for notifying members of where information came from and how it had been used in the past. -1 1 -3 1, 2, 3 KB/NHOLC The online learning community provided the opportunity for members to propose emerging project/investigation ideas that were relevant to their interests. 2 3 0 1, 2, 3 CoP The online learning community encouraged community members to apply information on the site to their own situations and questions. 0 3 -3 1, 2, 3 KB/NHOLC The online learning community highlighted and made clear the different groups / stakeholder perspectives involved in the project. -3 1 -1 1, 2, 3 CoP/NHOLC The online learning community provided the opportunity for community members to share the relevance of the projects to their lives. -2 2 0 1, 2, 3 CoP The online learning community’s overall shared purpose helped the project feel significant. 4 -3 1 1, 2, 3 FoK/NHOLC The online learning community provided members with the opportunity to share their knowledge of where they live and what they have experienced in their life. -4 -1 2 1, 2, 3 CoP Projects on the online learning community solved authentic, real-world problems. 4 1 4 1, 2, 3 FoK Each member of the online learning community brought knowledge to the community based on where they live and what they have experienced in their life. -2 0 3 1, 2, 3 KB/NHOLC The online learning community provided the opportunity to connect with members who had the expertise needed for an investigation. 2 -1 4 1, 2, 3 CoP The online learning community brought people together from different locations. 1 0 3 1, 2, 3 !28 The Journal of Community Informatics ISSN: 1721-4441 The NHOLC statements that defined each factor were of interest, as those characteristics had the potential to bridge the existing models of collaborative learning in the sociocultural learning theory literatures explored with the new emerging genre of the NHOLC experiences. The characteristics of each factor, consensus statements, and interview data were used to create descriptive titles and narratives of each perspective/factor. Using a participatory approach, the project coordinators of each case study were presented with and asked to give feedback and refinements on the factor descriptions and optimal Q-sorts during a one-hour long video-conference focus group. During the focus group participants were invited to share their interpretation of the factors as well and assist in further refining the factor titles, narratives, and implications for design principles. Their feedback is included in the factor descriptions and discussion sections below. CoP The online learning community provided the opportunity for community members to develop relationships with other members on the site. -1 -2 -2 Consensus CoP The goals of the online learning community are defined and refined by members. -3 -2 -2 Consensus CoP The online learning community connected individuals who use similar resources for work (same language, tools, experiences, definitions). -1 -1 -3 Consensus KB/NHOLC The online learning community provided members with various ways to connect with any member of the community. -2 -2 -2 Consensus CoP/NHOLC The online learning community helped community members connect to and work with members who had submitted information in the past. 0 -1 -1 Consensus CoP/NHOLC Starting with a shared purpose was important in generating trust amongst the various stakeholders. -1 -1 0 Consensus CoP The online learning community helped foster relationships and built trust among community members. 0 -1 1 Consensus CoP The online learning community provided the opportunity for community members to share information with one another. 2 2 1 Consensus CoP/NHOLC The different types of expertise present on the online learning community were a factor in making members feel like they were working toward the common goal of building knowledge together. 2 1 2 Consensus CoP The online learning community provided access to the tools and practices needed to solve authentic, real-world problems. 3 4 3 Consensus !29 The Journal of Community Informatics ISSN: 1721-4441 The factor analysis teased out three primary factors/perspectives that represent three distinct groups of participants from across the projects. Each group has their own distinct perspective on what is important to collaboration online for citizen science projects. The descriptions below explain how each distinct group viewed what was and was not important in their experiences of collaboration online in the citizen science project they participated in. Factor 1: Clarity of purpose is important, but not members’ backgrounds A shared purpose and a focus on real world problems foster collaboration online, while knowing community members’ backgrounds is not important. Demographic information: Factor 1 has 7 significantly loading participants and it explains 19% of the study variance. The participants in this factor represent participants in all case study sites, including 4 teachers, 1 scientist, and 2 general participants. Factor Interpretation: A clear understanding of and commitment to the project’s shared goal (114: +3) generates motivation (76: +3) for collaboration online. Collaboration online can be fostered by projects that solve authentic, real world problems (145: +4). A shared purpose of the collaboration can make the work of the project feel significant to participants (117: +4). Collaboration in an online learning community happens when participants have a commitment to building new knowledge that can be used by the whole community (180: +1) and a mechanism to critique and shape those new ideas that emerge from the community (114: +3). A structure within the online community to share responsibilities and decision-making (11: -3) was not of importance to online collaboration within the experiences of individuals in this factor. Bringing together diverse stakeholders (109: 0) or valuing the variety of expertise present in the community (111: -1) was not a driver to online collaboration or the development of roles in the online community (83: -3). Role development was not an ingredient for collaboration online. In addition, the ability to share historical and cultural knowledge (194: -2) or knowledge of where they live and what they have experienced in life (118: -4) relevant to the project with others was not part of this factor’s experience in fostering collaboration online. Factor 2: Diversity of perspectives and stakeholders matter, but shared goals are not needed Projects that are relevant to participants’ place and lived experience, as well as diverse participants that range across multiple perspectives are important to fostering online collaboration, while it is not as important for everyone to have one shared goal since everyone comes to the project for a somewhat different purpose. Demographic information: Factor 2 has 4 significantly loading participants and it explains 13% of the study variance. The participants are from Public Lab and WeatherBlur, !30 The Journal of Community Informatics ISSN: 1721-4441 including 1 project coordinator and 3 active participants (a fisherman and 2 local organizers). Factor Interpretation: Providing an opportunity for participants from multiple perspectives to build on the ideas of others to advance a project (174: +4) and encouraging diverse stakeholders no matter his/her age, expertise, or perspective to propose new questions (107: +2) is important in fostering online collaboration. To foster collaboration the online space attracted individuals to participate in and propose projects that are relevant to their everyday lives and interests (14: +1 and 57: +3). Knowing the different stakeholder groups involved in a project encouraged collaboration online (79: +1). Encouraging community members to apply information on the site to their own situations and questions fostered collaboration online (60: +3). The online space was a starting point for conversations amongst the various stakeholders (38: +3). Having shared goals/purpose or working toward building new knowledge together was neither a starting point, nor a uniting or motivating factor for collaboration online in the experience of these participants (186 & 92: -4 and 95: -3 and 138: -1). Factor 3: Building skills and trust amongst diverse stakeholders is important to collaboration, historical context of the collaboration is not This group believes that collaboration can move forward by sharing lived experiences and making new connections across boundaries. The ability to connect with individuals that had a diverse array of expertise and geographies in a safe supportive environment was important to online collaboration, whereas building on past historical knowledge of the community is not as important. Demographic information: Factor 3 has 5 significantly loading participants and it explains 17% of the study variance. The participants are active in both WeatherBlur and Vital Signs, including 1 teacher, 2 project coordinators, and 2 scientist participants. Factor Interpretation: Solving authentic real world problems (146: +4) and the ability or each member to share knowledge based on where they live and what they have experienced in their life (165: +3) fostered collaboration online. To foster collaboration it was important that the online space provided the opportunity for members to connect with others that had the expertise needed for an investigation (187: +4) and were from different locations (191: +3). To foster collaboration it was important that the online community provided the opportunity for members to move from new-comer to experienced members as they enhanced their skills and relationships on the site (81: +3) and that they felt freedom to express opinions and offer suggestions without fear of how the other members would judge it (127: +3). In the experience of these individuals the ability to track and understand how and why a project changed over time (28: -4), notifying members of where information came from and how it had been used in the past, or applying information on the site to their own situations and questions (60: -3) were not important in fostering collaboration. !31 The Journal of Community Informatics ISSN: 1721-4441 Consensus Statements Given that the goal of this study is to explore whether there are characteristics that participants considered essential design elements for collaboration in online learning communities, the statements that are agreed by all respondents to be important or not important across all participants are equally relevant (if not more so) than the factors themselves. Three statements were ranked as important by all factors, and are considered foundational in defining design principles for collaboration in online learning communities. The strongest consensus amongst the participants is the importance of the online space providing the access to tools (data collection protocols, research notes, maps, data analysis) and practices (user suggested improvements on how to collect data, how-to guides) needed to solve authentic, real-world problems. In addition, all participants identify the importance of being able to share information online with a wide array of expertise present in order to build knowledge together. Four statements were ranked as not important by all perspectives, and thus not considered imperative components for collaboration online. First and foremost, goals in the online community that are defined and refined by its members are not important to online collaboration. Connecting members who have similar skillsets, interests, experiences, and practices is not important to collaboration online. In addition, based on the interviewees’ experiences to date, 1) providing multiple ways to connect and 2) developing relationships with others on the site is not important to online collaboration. Implications for Designing Online Learning Communities Using the NHOLC Framework It is encouraging to see representation from each of the three cases in each factor. This cross-factor representation shows that there is enough consistency in each participant’s experience of the various online citizen science programs to uncover potential design principles that can work for all projects. The consensus statements indicate that all participants think that authentic, real-world problems and the tools necessary to solve them are essential for establishing online collaboration. A somewhat unexpected, but eagerly welcomed, emergent design principle is the importance of having an online community that brings together individuals from diverse stakeholder groups with vast areas of expertise (lived experience, skill sets, ages, jobs, etc.). Such as bringing representatives from a rural mining community together with technologists who can design low cost water monitoring equipment and electro-chemical engineers. Together this cast of unlikely characters that can only connect via an online community can define an potential issue in the rural community, develop a means for data collection when high end scientific equipment is not an option, analyze the data to !32 The Journal of Community Informatics ISSN: 1721-4441 understand what hard metals are in the water, and define a solution to mitigate the polluted water. In addition, consensus statements highlight that access to others with a similar skillset or interest to one’s own does not foster collaboration; this suggests that many areas of expertise and experience are necessary to solve the relevant real-world problems on which these projects focus. This finding emphasizes one component of the NHOLC framework - bringing together diverse stakeholders with a variety of expertise and lived experiences. Building on the community’s past knowledge is also a component of the original theoretical framework that varies across the factors. Factor 3 did not see building on the community’s prior knowledge as important, while the other factors were neutral about this component of the theoretical framework. These findings can be based on the fact that each of the case studies has a different level of prior knowledge available for participants to use. For example, the WeatherBlur project is new and does not have a large store of past data. Public Lab, on the other hand, has a glut of past knowledge about regional environmental monitoring projects and low-tech tools to use for data collection. Vital Signs has a valuable store of past knowledge and data but participants do not necessarily need to use it to complete their projects. Based on the findings of this study, building on past knowledge is not a key design principle of online collaboration but is worthy of further study. Participant-driven inquiry and decision making is a component of the original theoretical framework that was hotly contested among the factors. Statements related to shared goals, user-driven inquiry, shared responsibilities and decision-making fell all over the map. As Factor 1 highlights, a clear understanding and commitment to the shared goal of the project and user-driven inquiry is extremely important. Factor 2 can be interpreted to believe that it is not important for everyone to have one shared goal since everyone comes to the project for a somewhat different purpose and individual goal in mind. In juxtaposition, factor 3 considers shared goals and user-driven inquiry of neutral importance. This might be for a variety of reasons. Although each of the cases studied offer the ability for users to create new inquiries and define new goals, not every participant took advantage of that ability. Many participants expressed interest in doing more user-driven inquiry, but for a variety of reasons including time and low-confidence in their ability to do so, they did not take advantage of this design element in the online community. This low-confidence in participants’ abilities to do user-driven inquiry, but a high level of interest highlights a gap in the field that new citizen science initiatives should begin to address with professional development and added supports for user-driven inquiry. Within two of the cases, the democratization of science as driven by the public was a core founding philosophy while one of the cases did not start out with that ideal but has been moving toward user-driven inquiry in recent years. For all of these reasons, the theoretically driven design principle of user-driven inquiry and the evolution of shared goals should remain an important and highly valued design principle for online collaboration in citizen science but how they are operationalized in online environments needs further exploration and explanation. !33 The Journal of Community Informatics ISSN: 1721-4441 Reflecting on the findings of this study it is clear that the original NHOLC conceptual framework should be revisited and revised. The findings from the Q sort seem to suggest that emphasis should be heavily placed on the diverse participant groups component of the framework. The fact that participants can collaborate with individuals - who have new information relevant to their interests - that they may never have had the opportunity to connect with if it were not for the online community is an important and powerful driver in these communities. The findings also suggest that authentic real-world problems that are relevant for learners needs to be a highly prioritized component of successful online communities for citizen science. Across all of the projects and factors that emerged, real-world relevance was core to each individual’s experience. This emphasized component of the framework brings a new question into focus. Namely, how does an individual define or determine relevancy and real-world applications. In reflecting upon how important this component was to individuals, but yet somewhat undefined, highlights the need for further study to answer this emergent question. Image 3: Original Versus Revised NHOLC Conceptual Framework The three additional components – communication, sharing place-based data, and a shared purpose – were all very variable in levels of importance to collaboration in each factor. But each still played a key role in individual’s experiences. Additional research is needed to better understand the role of each of these components and how they could be revised to increase the impact of the NHOLC conceptual framework. !34 The Journal of Community Informatics ISSN: 1721-4441 Limitations of the study The design principles presented here could be viewed as “common sense” and nothing new to the field. However, simply having a starting point that filters out the other potentially important design principles is an advancement that did not exist prior to this study. Many researchers across the country are asking similar questions and the design principles outlined here provide productive pathways to address these lines of inquiry. In addition, the findings presented rely upon members of the community that have positive intentions as they populate the databases, comment, discuss, and share information. Without strong values in the online community and the drive to self-monitor content these online communities intended for positive community change and understanding could quickly devolve into unsafe learning contexts. This study is not without limitations. First, the case studies were all chosen based on whether or not the projects showcased the core characteristics of a NHOLC. This best-case scenario sampling method provides a focused lens to the study, but it also places limitations on the findings of the design principles. Therefore, one must keep an open mind to additional design principles that do not fall within the NHOLC framework. The study at hand emphasizes non-hierarchical models to understand how those specific environments function. Therefore it does not explore the ways that a more hierarchical model may contribute to online collaboration and local environmental action. It is the hope that additional research studies will simply use the findings of this study to ask additional research questions, perhaps by applying new theoretical and conceptual frameworks as lens to highlight design principles. A second limitation of the study comes from the sampling of the interviewees. The sampling of project participants is not random. The study asked each project coordinator to suggest potential interviewees. The study incorporates individuals that had been some of the most active in the online communities and could hopefully provide the greatest insight into how and why it worked for them. Due to this method the findings of this study does not include the perspectives of individuals that joined the communities and dropped out, those that were not highly active, or those that did not want to participate in the interview process. Linking the NHOLC Conceptual Framework Back to the Future of Citizen Science This paper explores how sociocultural learning theory can inform design principles for online citizen science learning communities in order to inspire local environmental action. To answer this question the research presented applied the new interwoven Non- hierarchical Online Learning Community (NHOLC) framework to a multiple case study of !35 The Journal of Community Informatics ISSN: 1721-4441 three citizen science projects that use online learning communities as a core component of their program. Past research in citizen science contexts has shown that individuals have greater motivation to engage and learn if the topics being explored are relevant to their everyday lives (Falk, 2001; Dierking, 2010). Individuals gain additional motivation if they can directly affect the learning process, content, or outcomes/actions (Bonney et al. 2009; Falk 2001). In addition, the ability of participants to have the opportunity to do the work of scientists as they experience the same thrills of inquiry, debate, and new questioning that happens during true scientific inquiry relevant to their interests (Bonney et al., 2009) is an extremely valuable experience. The findings of this study certainly align with this prior research, but it also adds some fresh new insights into how online spaces specifically can be structured to enable collaboration between participants in online communities for citizen science. The findings represented here present the theoretical foundations, conceptual framework, and essential design principles for online citizen science projects. The findings provide a starting point for researchers and practitioners to further develop this area of work. Findings from this study suggest, the design elements of the NHOLC framework that rise to the top as important design elements for use in online learning communities for citizen science are: 1. Access to tools (data collection protocols, research notes, maps, data analysis) and practices (user suggested improvements on how to collect data, how-to guides) needed to solve authentic, real-world problems; and 2. Diverse stakeholder groups from vast areas of expertise (both professional and lived experiences). The other elements of the NHOLC framework are important to collaboration for specific participant types and are worthy of additional study related to how to foster or challenge collaboration online within specific contexts (e.g. targeted audiences). In addition, further research is needed to understand exactly what forms or tools and practices are needed to solve problems, as well as foster collaboration and local action. Further research is also needed to explore the range and variation of the NHOLC framework components by diving deeper into participants’ experiences of the projects, with special attention to the technological functions and programmatic components of the online community, and how they supported the participant experience and informed participants’ responses to the study presented above. For instance, which technological supports in the online community provided a participant with access to diverse stakeholders and how they were used. This instrumental case study combines the lessons learned across three innovative online citizen science projects that have all been successful in fostering localized environmental actions. The NHOLC framework serves as a lens with which to better understand the !36 The Journal of Community Informatics ISSN: 1721-4441 structural make up of the online functions and the experiences of the participants. In addition, the NHOLC framework and its associated design principles empower citizens with data, tools, and the necessary networks to find solutions to the environmental questions they have about their own communities. It is hoped that the findings of this study will contribute to the design of other citizen-based online communities that want to leverage the power of our modern digitally connected society to solve local and global environmental problems Acknowledgements This research was funded by National Science Foundation’s Cyberlearning: Transforming STEM Education program (award #1451315). Special thanks to Dr. Karen Peterman and Dr. Sue Allen for their guidance and assistance in the development of this study and the Q Sort statements. Special thanks are also in order to Dr. Janet Kolodner, Dr. Christopher Hoadley, Dr. James Karlan, Dr. Jean Kayira, Dr. Mike Mueller, and David Sobel for their mentoring, advice, and support throughout the entire process. References Barry, J., & Proops, J. (1999). Seeking sustainability discourses with Q methodology. Ecological Economics, 28(3), 337–345. https://doi.org/10.1016/ S0921-8009(98)00053-6 Brewerton, P. M., & Millward, L. J. (2001). Organizational Research Methods: A Guide for Students and Researchers. SAGE. Brown, S. (1980). Political Subjectivity: Applications of Q methodology in political science. Yale University Press. Dara O’Neil. (2002). Assessing community informatics: a review of methodological approaches for evaluating community networks and community technology centers. Internet Research, 12(1), 76–102. https://doi.org/10.1108/10662240210415844 Eagle, D., Hague, B., Keeble, L., & Loader, B. D. (2005). Community Informatics: Shaping Computer-Mediated Social Networks. Routledge. Gonzalez, N., Moll, L. C., & Amanti, C. (Eds.). (2005). Funds of Knowledge: Theorizing Practices in Households, Communities, and Classrooms (1 edition). Mahwah, N.J: Routledge. Green, S. B., & Salkind, N. J. (2010). Using SPSS for Windows and Macintosh: Analyzing and Understanding Data. Prentice Hall Press. Retrieved from http://dl.acm.org/ citation.cfm?id=1894956 Lave, J., & Wenger, E. (1991). Situated Learning: Legitimate Peripheral Participation (1st edition). Cambridge England ; New York: Cambridge University Press. McKeown, B., & Thomas, D. (2013). Q Methodology. SAGE Publications, Inc. Moll, L. C., Amanti, C., Neff, D., & Gonzalez, N. (1992). Funds of knowledge for teaching: Using a qualitative approach to connect homes and classrooms. Theory Into Practice, 31(2), 132–141. https://doi.org/10.1080/00405849209543534 !37 The Journal of Community Informatics ISSN: 1721-4441 Scardamalia, M., & Bereiter, C. (2006). Knowledge building: Theory, pedagogy, and technology. The Cambridge Handbook of the Learning Sciences, 97–115. Schusler, T. M., Krasny, M. E., Peters, S. J., & Decker, D. J. (2009). Developing Citizens and Communities through Youth Environmental Action. Environmental Education Research, 15(1), 111–127. Smith, A., G., Sobel, D. Sobel, D. (2010). Gregory A. Smith,David Sobel’sPlace- and Community-Based Education in Schools (Sociocultural, Political, and Historical Studies in Education). Routledge. Sobel, D. (2005). Place-based education: connecting classrooms & communities. Great Barrington, MA: Orion Society. Stephenson, W. (1935). Technique of factor analysis. Nature, 136, 297. https://doi.org/ 10.1038/136297b0 Stoecker, R. R. (2005). Is Community Informatics Good for Communities? Questions Confronting an Emerging Field. The Journal of Community Informatics, 1(3). Retrieved from http://www.ci-journal.net/index.php/ciej/article/view/183 Wenger, E. (2000a). Communities of Practice and Social Learning Systems. Organization, 7(2), 225–246. https://doi.org/10.1177/135050840072002 Wenger, E. (2000b). Communities of Practice: Learning, Meaning, and Identity (1 edition). Cambridge, U.K.; New York, N.Y.: Cambridge University Press. Yin, R. K. (2014). Case study research: design and methods. !38 Introduction Defining the NHOLC Conceptual Framework Strengths of the NHOLC Framework Methods – Identifying the Cases Q – Methodology Descriptions for Case Study Sites Development of Q- Statements Data Collection Defining Factors: Perspectives on what was most important for online collaboration and on-the-ground action Factor 1: Clarity of purpose is important, but not members’ backgrounds Factor 2: Diversity of perspectives and stakeholders matter, but shared goals are not needed Factor 3: Building skills and trust amongst diverse stakeholders is important to collaboration, historical context of the collaboration is not Consensus Statements Implications for Designing Online Learning Communities Using the NHOLC Framework Limitations of the study Linking the NHOLC Conceptual Framework Back to the Future of Citizen Science Acknowledgements References