The Australasian Journal of Information Systems (AJIS) began in 1993 and was based on the simple realisation that there was a need for an academic ‘outlet’ for IS research in Australia Australasian Journal of Information Systems Volume 17 Number 2 2012 109 THEORIZING IN INFORMATION SYSTEMS RESEARCH USING FOCUS GROUPS France Belanger Virginia Tech belanger@vt.edu ABSTRACT Information Systems researchers have embraced a number of qualitative research approaches and methodologies, including interviews, observations, and even action research. One research method gaining visibility in IS research is the focus group research method. Focus groups have the potential to provide great insights into phenomena of interest to IS researchers as they allow researchers to get deeper into the topic of interest by providing more background information about the circumstances of the subject’s answers or opinions. This paper presents a review of focus group research in the information systems literature, and provides a discussion of how and when the focus group research method can be the most appropriate method to use for IS theorizing. The discussion highlights the idea that the focus group research method is particularly useful for exploratory research on topics where concepts normally emerge through interactions among individuals or where concepts are initially unclear to participants, and as part of a multi-method research program for theory development. Examples of focus groups used in theory development are provided, together with a discussion of the limitations of the research method. Keywords: focus group, qualitative research, information systems research, theory development INTRODUCTION “A research process can be viewed as a series of interlocking choices, in which we try simultaneously to maximize several conflicting desiderata”(McGrath 1982) Information systems researchers have increasingly embraced new research approaches, methodologies, and epistemologies over the years. Each approach or perspective provides its own set of advantages and disadvantages. Qualitative techniques, for one, allow researchers to describe and explore an area of study in a rich context, in order to gain an understanding of the subjective experiences of the participants (Bryman 1988). Today, one can often find studies using qualitative methodologies published in leading IS journals. Among qualitative research methods, one often reads of case studies used for grounded theory or model testing, where interviews and observations have dominant roles to play. One research method gaining visibility in IS research is the focus group research method (Sobreperez 2008). Rooted in market and social research, focus groups represent one qualitative research method where participants are selected and brought together to explore and discuss a specific topic, often in great detail (Morgan 1997). Importantly, in focus groups, participants are able to interact with, and react to, other participants in responding to questions and prompts of the moderator (Krueger and Casey 2000). It is argued in this paper that these interactions can be extremely useful in information systems theory development for phenomena where constructs exist in or emerge from individuals’ interactions with one another, such as team software development or impacts of telecommuting on groups. mailto:belanger@vt.edu Australasian Journal of Information Systems Volume 17 Number 2 2012 110 Focus groups not only allow researchers to collect a concentrated set of interactions in a given time period, they also enable researchers to obtain more background information about the circumstances of the answers provided by the subjects (Krueger 1994). As a result, focus groups have the potential to provide significant insights about phenomena of interest to IS researchers in several situations: first, when information systems researchers are interested in new topics to be studied via exploratory research (e.g., what are the critical success factors for implementation of cloud computing); second, focus groups can also support research where constructs emerge from group interactions as opposed to individuals situations (e.g., how do software development teams benefit from (or not) mandatory use of new standards) (Krueger and Casey 2000; Le Rouge and Niederman 2006); and finally, a review of the IS literature reveals that focus groups can play an important role in developing theoretical models of IS phenomena to be tested. The purpose of this paper is therefore to explore the use of the focus group method in the information systems literature, discuss its advantages and limitations, and discuss the applicability of this research method for the development of theories in information systems. More specifically, the discussion is guided by the following question: Under which circumstances is the focus group research method most appropriate for IS theorizing, and how can the method best be used for IS theorizing? This paper provides two main contributions: first, it offers a review and categorization of how the focus group research method has been used in information systems research, and second and more importantly, it discusses how and when the focus group research method can be used for theory development in IS research. The remainder of the manuscript is organized as follows. The background section briefly discusses the focus group research method followed by a review and analysis of focus group research in the information systems literature. Then, a discussion of the applicability of the focus group research method to the field of information systems and the potential role of focus groups in IS theorizing are provided and illustrated by research examples. Finally, limitations of this work are offered before concluding comments are provided. THE FOCUS GROUP RESEARCH METHOD The focus group research method has long been used in studies of consumer attitudes, perceptions, and opinions in the marketing and business communications literature (Fern 1982; Hartman 2004). As a result, many researchers describe the method in substantial details (Edmunds 1999; Fern 2001; Krueger 1994; Krueger and Casey 2000). This section provides a brief overview of the method, with more details provided in Appendix A. The focus group method is a qualitative data gathering technique where focus group sessions involve several participants assembled for a planned discussion to explore a specific topic of interest to researchers in a permissive, non-threatening environment (Krueger 1994). The focus group research method benefits from the interaction among participants, which can reveal shared ideas, reactions, and opinions on the topic of the study. Morgan (1997) refers to this as collaborative construction instead of consensus or negotiation. The participants are therefore influenced by each other when responding to ideas and comments in the discussion. In reality, they can also be influenced by the moderator, the setting, or any tools used in conducting the focus group. As a result, focus group research often better fits in the interpretivist epistemology (Myers 1997), where participants’ interactions influence each other and might influence the interpretation of the researcher, allowing that participants might be influenced by their interactions with the researcher/moderator. Without the interaction among participants, findings represent individual perceptions, i.e., ideas in isolation, as opposed to shared perceptions or individual perceptions embedded in a social or problem context. Indeed, Morgan (1997) suggests that the synergistic effects of focused interactions, can provide greater insights than the sum of individual interviews. Australasian Journal of Information Systems Volume 17 Number 2 2012 111 Typically, focus group sessions are conducted by a moderator with the responsibility of running the session. A very high level of involvement is necessary to ensure that all participants voice their ideas, attitudes, perceptions or opinions, and that emerging concepts of interest are explored through further probing when appropriate (Sobreperez 2008). The focus group participants are selected because they are knowledgeable about the topic investigated. The sessions are recorded with audio and/or video equipment, and are then transcribed so that they can be analyzed. In that aspect, focus groups follow similar procedures as other qualitative data collection approaches such as interviews and observations (Strauss and Corbin 1998). The focus group transcripts can be analyzed using standard qualitative analysis software and techniques (Miles and Huberman 1994). The results can be used to develop grounded theories (Eisenhardt 1989), provide contextual data for case studies (Yin 1994), or more frequently, to provide exploratory constructs and relationships that will be tested at a later stage of research (e.g., Dickinger et al. 2008; Lee et al. 1995). Interestingly, the data gathered from focus groups are not limited to transcripts obtained from the discussions in the sessions, but can also include the moderator (or assigned scribe) notes, body language if video is used, results from debriefing sessions (if conducted), and any pre-session questionnaires that can be distributed at the beginning of focus groups (Edmunds 1999). Theory building is important to the development of the IS field, and as Straub (2009) suggests, having strong theoretical foundations and contributions is a major factor in publishing in top journals. Indeed, Tams and Grover (2010) find that factors related to theoretical development, rather than research methodology or implications, lead to more highly cited, i.e., influential, articles in IS literature. Given the possibility of eliciting constructs from collaborative construction (constructs emerging from group discussions) (Morgan 1997) and the ability to triangulate verbal data with other data sources such as gestures and reactions, this research method could be uniquely useful for theory building in many IS contexts. For example, when the domain of interest is exploratory and the focus groups are used to generate information or ideas that can be later tested, when trying to obtain a deeper understanding of constructs (i.e. their dimensions) in established theory, or, when the constructs of interest exist in or emerge from the interactions of individuals with one another. This is particularly important to the information systems field because many topics of interest in the domain, from knowledge management, to telecommuting, to software development are affected by individuals’ interactions. Using an individual unit of analysis to study group performance might lead to less complete and realistic theories. In order to explore this possible issue, and before further exploring how and when the focus group research method can be used in information systems theory development, the IS literature is reviewed to identify how focus group research has been used in the IS field. LITERATURE REVIEW: FOCUS GROUP RESEARCH IN IS Kruger (1994) suggests that focus groups can be very useful in gaining insights into a research domain in two particular cases: when there is limited research previously published, or when the focus group is part of a larger scale research effort. Others suggest that focus groups have broad appeal for various types of evaluations of products, concepts or services, or to generate ideas for questionnaires (Edmunds 1999). Consistent with recent searches of the literature in top IS research journals (Baskerville and Myers 2009; Bélanger and Crossler 2011; Wang 2010), two well-known databases of articles in business, ABI/Inform and Business Source Complete, were used to search for peer-reviewed journal articles in information systems that use the focus group research method as part of their research design between 1990 and 2011, using the keywords “information system” and “focus group”. A total of 94 articles were identified in ABI/Inform and 72 in Business Source Complete. After screening for IS journals Australasian Journal of Information Systems Volume 17 Number 2 2012 112 (broadly defined to include information science, computer science and related fields), there were 58 articles from ABI/Inform and 51 from Business Source Complete. Taking into account articles found in both databases, 75 individual articles were downloaded for further review. In the second round of review, only papers that study an IS phenomena using focus groups as part of the research design were retained. Papers removed from the list included papers that suggest future work should include focus groups, that were not IS-related, or papers that discussed focus groups as a method to be adapted for other purposes such as online market research (Clapper and Massey 1996; Markham et al. 2006; Montoya-Weiss et al. 1998) or for applicability checks (Glass 2009; Rosemann and Vessey 2008), discussed focus group as a research method (Sobreperez 2008; Stahl et al. 2011), discussed focus group as a method for requirements determination in IS development (Leifer et al. 1994; O'hEocha et al. 2010; Peffers and Tuunanen 2005) or to inform design research (Tremblay et al. 2010), or explored IS education issues using focus groups (Dunning et al. 2001; Dwyer and Knapp 2004; Maheshwari et al. 2009; Shuk Ying et al. 2008). The resulting sample of 58 papers shows that the oldest papers date to 1995 and 1996 in MIS Quarterly (Lee et al. 1995; Smith et al. 1996), Information Systems Research (Manning 1996), and Information Technology and Libraries (Tillotson et al. 1995). The sampled papers are summarized in Table 1. Following the types of focus groups identified by Kruger (1994), information systems papers were classified along the dimensions of sole method or multi-method (also called mixed-method). Sole method papers are those where the entire research findings of the paper are based on the use of the focus group research method. The multi-method papers represent those studies where the focus group method is used in conjunction with other research methods. Extending the work of Kruger (1994), the classification is further broken down into exploratory vs. explanatory research. Exploratory research focuses on “what is” the state of some IS phenomenon, whereas explanatory research focuses on “how” and “why” some IS phenomenon is happening. The distinction between the use of interviews and the use of focus groups (sometimes called group interviews) is important and sometimes misunderstood. In conducting the review of the literature, one study was found (Martz et al. 2007) where the researchers use the terminology of focus group in the description of the study (in the abstract) without apparently using focus groups as their research method. The authors conducted 18 interviews of practitioners and academics (in addition to observations) with the purpose of designing a Master of Healthcare Informatics curriculum. However, the interviews appear to be individual, which would not constitute a focus group (the focus group terminology is not used except in the abstract). Focus Group Type Citation Topic Sample and other Methods (for multi- method papers) Focus Group Purpose Sole method: exploratory (Campbell et al. 2005) – Journal of Enterprise Information Management Alignment of business and IS 3 focus groups with 6 IS/IT managers in each group Explore how managers view the alignment between business and IT. Participants develop a causal loop diagram. (Grant and Fitzgerald 2005) - Electronic Journal of Business Research Methods Nexus between teaching and research in IS 2 focus groups with staff and educators at universities Explore whether a nexus exists between teaching and research at universities. Also, evaluate focus group as method to explore this. (Huang and Lin 2009) – Journal of Computer Information Systems Email usage 1 focus group with 11 participants Explore email usage of knowledge workers. (Jarvenpaa and Lang 2005) - Information Systems management Mobile technology use 33 focus groups with 222 participants in 4 countries (Finland, Japan, China, USA) Identify paradoxes and behaviours of mobile technology users. Develop a user-technology interaction Australasian Journal of Information Systems Volume 17 Number 2 2012 113 Focus Group Type Citation Topic Sample and other Methods (for multi- method papers) Focus Group Purpose process framework. (Jarvenpaa et al. 2003) - Communications of the ACM Mobile technology use 32 focus groups with 200 participants in 4 countries (Finland, Japan, China, USA) Identify services used by mobile technology users, and issues related to use of mobile IT. (Le Rouge and Niederman 2006) - Communications of the AIS Public health knowledge management 6 focus groups with higher education administrators, educators and students Use focus groups for requirement elicitation to develop public health knowledge base system. (McKeen et al. 2005) - Communications of the AIS Digital dashboards 1 focus group with IT managers (# of participants unknown) To explore the use and types of digital dashboards in use by IT managers. (McKeen and Smith 2006) - Communications of the AIS Technology roadmap 1 focus group with IT managers (# of participants unknown) Explore managers’ practices in terms of creating and evolving a technology roadmap. (McKeen and Smith 2007) - Communications of the AIS IT delivery options 1 focus group with IT managers (# of participants unknown) Identify decision criteria for IT delivery to build a decision framework to guide selection of delivery options for IT. Sole method: exploratory (continued) (McKeen et al. 2009) - Communications of the AIS IT staffing 1 focus group with IT managers (# of participants unknown) To explore the current IT staffing challenges and issues. (McKeen and Smith 2010b) - Communications of the AIS Total cost of ownership 1 focus group with IT managers (# of participants unknown) Identify firms’ use of total cost of ownership approach in IT. (McKeen and Smith 2010a) - Communications of the AIS Application portfolio management (APM) 1 focus group with IT managers (# of participants unknown) To understand managers’ use of APM strategies and practices. (Payton and Zahay 2003) – Journal of database Marketing CRM implementation success 5 focus groups with 21 marketing, IS, and strategic managers in a healthcare firm Identify factors that lead to CRM implementation success for the focal firm (healthcare payer). (Raitoharju et al. 2009) – Electronic Journal of Business Research Methods Health Information Systems Focus groups (# unknown) with 23 healthcare professionals. Examine usage of health information exchange systems in 2 kinds of healthcare sectors (i.e. Finland and USA) (Reid et al. 2010) - European Journal of Information Systems Challenges for women in IT 6 same-sex focus groups (3 each) with 45 IT professionals in 3 USA companies Identify how men and women describe the unique challenges that women face in the IS workplace. (Resatsch et al. 2008) – Electronic Markets Point-of-sale RFID 2 focus groups with 10 consumers and 10 sales assistants Identify factors determining intentions to use RFID-based smart products (Scott et al. 2009) - Communications of the AIS Information systems as a career 6 focus groups with 31 undergraduate students Identify factors that influence students to select IS as a career. (Smith and McKeen 2003) - Communications of the AIS Knowledge management usage 1 focus group with knowledge managers (# of participants unknown) from 25 companies Explore the state of knowledge management usage and its likely future development. (Smith and McKeen 2004) - Communications of the AIS Knowledge management marketing 1 focus group with practicing knowledge managers (# of participants unknown) Identify how managers market knowledge management (KM) in organizations to develop a framework for marketing of KM within the organization. Australasian Journal of Information Systems Volume 17 Number 2 2012 114 Focus Group Type Citation Topic Sample and other Methods (for multi- method papers) Focus Group Purpose Sole method: exploratory (continued) (Smith and McKeen 2007) - Communications of the AIS Information management implementation 1 focus group with IT managers (# of participants unknown) Identify issues and challenges in information management. (Smith et al. 2007) - Communications of the AIS Perceptions of IS in organizations 1 focus group with IT managers from 15 companies Identify how IS is perceived in companies – compared with similar results from 1996 study. (Smith and McKeen 2010a) - Communications of the AIS Business IT relationship 1 focus group with senior IT managers (# of participants unknown) Explores the relationship between IT and business and how it can be characterized. (Smith and McKeen 2010b) - Communications of the AIS Business and interpersonal competencies of IT staff 1 focus group with senior IT managers (# of participants unknown) To explore the business and interpersonal competencies that IT staff will need in order to do their jobs effectively over the next five to seven years and what companies should be doing to help develop them. (Smith and McKeen 2011) - Communications of the AIS Collaboration and IT 1 focus group with IT managers (# of participants unknown) Identify the role of IT in enabling collaboration within organizations. Sole method: explanatory (Sutton et al. 2008) - Journal of the Association for Information Systems Critical risk factors in B2B e-commerce relationships 5 focus groups: 3 internal groups (IS security, internal IT audit, e-commerce development managers) and 2 external groups (e- commerce consultants, external IT auditors) Identify critical risk factors used to assess impact of B2B e- commerce on overall enterprise risk. Tests performed between groups. Multi-method: exploratory (Bishop et al. 2000) – Journal of the American Society for Information Science and Technology Use of digital libraries initiative 3 focus groups with 17 participants to supplement information from interviews, observations, usability testing, logs, and surveys. Explore the use of the Digital Libraries Initiative at the University of Illinois (Chesney et al. 2009) - Information Systems Journal Grieving behaviours in virtual worlds 4 online focus group with 14 participants. Combined with 15 hours of non-participant observations Identify grieving behaviours in virtual worlds, their impact, why they occur, who the likely targets are, and who perpetrators are. Multi-method: exploratory (continued) (Cooper and Heinze 2007) – Journal of Information Systems Education Multi-year projects in IS education programs (# unknown) focus groups with students and staff, combined with interviews and survey Identify best ways to evaluate multi-year projects in IS education programs. (Hill et al. 1998) – Journal of Global Information Management Information technology transfer and culture (# unknown) focus groups to supplement interviews and a field study in five Arab countries Explore linkages between information technology transfer and sociocultural factors that support or impede a successful transfer. (Jones et al. 2008) - Communications of the AIS Post-installation use of ERP 5 focus groups in 1 organization to supplement 104 survey responses from five other organizations Determine level and effectiveness of ERP training and usage several years after implementation. (Kelly et al. 2007) – Analytical 2 focus groups with 4 Identify evaluation criteria for Australasian Journal of Information Systems Volume 17 Number 2 2012 115 Focus Group Type Citation Topic Sample and other Methods (for multi- method papers) Focus Group Purpose Journal of the American Society for Information Science and technology Question- Answering Systems analysts, followed by individual interviews interactive, analytical question- answering (QA) systems. (Kesner 2008) – Communication of the AIS Competencies required of students 4 groups and 14 interviews with approximately 17 employees (15% of survey population), combined with survey of 111 employees Identify competencies IT students need as perceived by potential employers. (Klaus and Blanton 2010) - European Journal of Information Systems User resistance 1 focus group of 7 IT professionals and 34 interviews in 3 organizations Identify the determinants of user resistance to IT implementation. (Lehaney et al. 1999) – Journal of End User Computing Database marketing 1 focus group to supplement survey Identify the success factors for technology-enabled database marketing in the tourism industry. (Manning 1996) - Information Systems Research Use of cellular technology in police context (# unknown) focus groups of police officers and their leaders, combined with interviews and observations Explore use of IT in police context. (Nahar et al. 2006) – Information & Management Information technology transfer 16 (or more?) focus groups of 4 to 6 participants (complemented by interviews) of IT managers, general managers, and executives in each in 4 Finnish companies Identify and rank the various factors affecting IT-supported international technology transfers. Rankings of factors quantitatively evaluated between firms. Multi-method: exploratory (continued) (Otondo et al. 2009) – European Journal of Information Systems Adoption of RFID 2 focus groups with 21 participants, complemented with data from published interviews Explores how managers evaluate RFID and participate in its adoption. (Pettigrew et al. 2002) – Journal of the American Society for Information Science and Technology Use of library community network systems (# unknown) focus groups of public library staff, human service providers and users, complement surveys and interviews Identify how users use public networked community information systems, as well as the barriers and benefits of use. (Ramesh and Jarke 2001) – IEEE Transactions on Software Engineering Reference models for requirement traceability 37 focus groups (several phases) in 26 software development firms, combined with interviews Identify reference models for requirement traceability and discuss requirements for future traceability mechanisms. (Srivastava and Teo 2009) – Communications of the AIS Trust in e- government 5 focus groups with 45 participants in Singapore, followed by 27 interviews, and use of secondary data. Identify factors determining used trust in e-government. (Tillotson et al. 1995) - Information Technology and Libraries Use of library for Internet usage 2 focus groups of 17 undergraduates to supplement online survey and logs of telnet Explore use of the Internet via the University of Toronto Library online catalog and information system Australasian Journal of Information Systems Volume 17 Number 2 2012 116 Focus Group Type Citation Topic Sample and other Methods (for multi- method papers) Focus Group Purpose connections . (van Velsen et al. 2008) – International Journal of Enterprise Information Systems Requirements elicitation for an ERP system 1 focus group with 6 users of ERP, compared (findings) with heuristic evaluation of the interface by experts and developers Identify most important functions and tasks for ERP users. Multi-method: concept generation for testing (explanatory) (Dickinger et al. 2008) – European Journal of Information Systems Adoption of Push to Talk (PTT) technology 3 focus groups with 23 experts, preceded by 10 interview with experts, followed by survey to test Identify factors that affect user intentions to use PTT technology. (Galup et al. 2008) - Journal of Computer Information Systems Impact of job characteristics on satisfaction 6 focus groups in a governmental agency (county) in Florida followed by survey and interviews Generate constructs for model of satisfaction (stage 1 of 3). (Goh et al. 2010) – Journal of Computer Information Systems Mobile tourism 1 focus group with 8 participants to generate items for a survey administered to 210 respondents Identify mobile services of importance to tourists. (Khalifa and Limayem 2003) - Communications of the ACM Internet shopping (# unknown) focus groups with 177 Internet consumers (undefined) followed by survey Identify factors that affect adoption and use of Internet shopping. Multi-method: concept generation for testing (explanatory) (continued) (Krasnova et al. 2010) – Journal of Information Technology Online social networks 2 focus groups with 16 students, followed by survey of 259 students Generate items to measure students’ determinants of intentions to disclose information. (Lee et al. 1995) - MIS Quarterly Skills and knowledge requirements for IS professionals 50 IS practitioners and academics (unknown # of focus groups), followed by survey Generate items for subsequent survey of IS trends. (Smith et al. 1996) – MIS Quarterly Concern for information privacy (# unknown) focus groups with 18 consumers (some interviewed ; some in focus group) and (# unknown) focus groups with 25 employees. Combined with interviews; followed by survey. Generate and validate items for development of CFIP instrument. (Torkzadeh et al. 2006) – Decision Support Systems Customer relationship management 6 focus groups of 6 customer service representatives, followed by survey with 1460 responses Identify key barriers to success of customer relationship management. (Wang et al. 2004) – Information Systems Frontiers Service quality, customer value, satisfaction (# unknown) focus groups (# unknown participants) Identify and refine items to measure service quality beyond the SERVQUAL instrument. (Weidong and Lee 2005) – Journal of Management Information Systems Information systems development project complexity (# unknown) focus groups with 45 IS managers followed by survey Generate items to measure ISD project complexity (phase 1 of 4). Multi-method: (Fitch 2007) – Evaluation of a 8 focus groups with 65 Evaluate usefulness of Australasian Journal of Information Systems Volume 17 Number 2 2012 117 Focus Group Type Citation Topic Sample and other Methods (for multi- method papers) Focus Group Purpose concept generation after testing or framework validation (explanatory) Decision Support Systems risk assessment instrument practitioners to supplement survey data (focus groups conducted with subset of survey respondents) information obtained from a risk assessment decision support tool developed by the authors. (Price and Shanks 2005) – Journal of Information Technology Information quality 3 focus groups with IT practitioners, IT academics, and end users to validate framework Develop a framework for information quality then validated with focus group results. (Xia and Lee 2004) – Communications of the ACM IS development projects Focus groups and interviews with 74 development managers, followed by survey of 549 managers to validate framework Develop a framework to explain IS development project complexity, which is then validated by focus groups, interviews and a survey. Multi-method: concept generation after testing or framework validation (continued) (Xu et al. 2011) – Information Processing and Management Intelligent agent systems for executive information 3 focus groups with 31 middle and top-level managers, followed by 25 interviews to test hypotheses Proposes 4 hypotheses that are validated with focus groups and interviews. Multi-method: Others (Chang Lee and Kwon 2008) – Decision Support Systems Map-driven avatar design decision support system 1 focus group with 9 avatar experts (together with brainstorming sessions) Uses focus group to extract the cognitive maps of experts and create a collective cognitive map. (Smith et al. 2010) – MIS Quarterly Mandated compliance to information security standard 22 focus groups (unknown # participants) as interventions between 8 online surveys (with interview, forums and discussions as other interventions) Studies adoption of information systems security (ISS) compliance in government. Focus groups viewed as action research. Table 1. Information Systems Journal Articles Using The Focus Group Research Method USING THE FOCUS GROUP RESEARCH METHOD FOR IS THEORIZING The number of studies and the quality of journals publishing research using focus group research methods in the sample clearly shows that focus group research has a substantive role in the repertoire of research methods for IS researchers. The following discussion provides an analysis of the findings from Table 1, and further explores how and when the focus group research method can be appropriate in IS theorizing. Table 1 reveals a number of interesting findings regarding the use of the focus group research method in information systems. First, it is clear that there are quite a few articles published that use this research method in general. Second, one can notice that most studies using the focus group research method have been published in recent years: 49 of the 58 studies were published between 2003 and 2011. This suggests that the method is gaining acceptance as an appropriate research method in information systems research. More importantly, the categorization of papers along types of focus group research reveals some interesting trends, which are discussed below, together with the role of each category of focus group research method in IS theorizing. Before the findings from Table 1 are discussed within the context of IS theorizing, it is important to define the concepts related to theory. Straub (2009) suggests that research rarely develops new Australasian Journal of Information Systems Volume 17 Number 2 2012 118 theory, but instead focuses on adapting or refining existing theories. Yet, theory development is core to the growth of a research discipline with many and various phenomena of interest. In discussing the role of theories in information systems, Gregor (2006) identified fives types of theories: analyzing, explaining, predicting, explaining and predicting, and design and action. Importantly, different types of theories can provide different insights into an information systems domain. Table 2 summarizes the different types of theories, and suggests how focus groups inform theory development for each type of theory. The subsequent discussion of each category of focus group studies will offer further examples of how they can be used in IS theorizing. Theory Type Definition Sample Use of Focus Groups Examples Using Focus Groups in IS Analyzing Describe the state of information systems phenomena. Elicit knowledge about a research topic from domain experts in a series of focus groups. Jarvenpaa and Lang (2005) identify services used by mobile technology users, and issues related to use of mobile IT. Explaining Explain what is occurring but do not provide testable predictions. Identify and rank important factors explaining usage of an IT via focus groups of managers. Nahar et al (2006) identify critical success factors for IT transfers and rank them across firms. Predicting Provide testable predictions without well-developed causal relationships. Use focus groups to elicit possible outcomes from an IS. Campbell et al (2005) explore how managers view the alignment between business and IT. Explaining and Predicting Explain what is occurring and provide testable predictions with causal explanations. Use focus groups to elicit items for further testing and/or to develop a testable model. Smith et al (1996) generate and validate items for development of the Concern for Information Privacy instrument. Design and Action Design an IT artifact or a framework to evaluate such artifacts. Elicit requirements for the development of an artifact; for example, showing a prototype to focus groups for feedback. LeRouge and Niederman (2006) identify architectural requirements for a public health knowledge system. Table 2. Focus Groups and Theory Types (adapted from (Gregor 2006)) As stated in the prior discussion, the focus group research method can be useful is IS theorizing when ideas can be generated from group interactions or when constructs exist in or emerge from the interactions of individuals with one another. Different from interview data, focus group data allow Australasian Journal of Information Systems Volume 17 Number 2 2012 119 researchers to understand the extent to which factors are perceived similarly and/or the extent of shared meaning and relative value of the constructs. FOCUS GROUP AS SOLE METHOD - EXPLORATORY An important finding from Table 1 is that the majority of papers using the focus group research method in information systems fall in the sole method – exploratory category (25 of 58). This is interesting not only because it is consistent with the type of research for which focus group research methods are recommended (Krueger 1994), but also because it suggests that many more research projects could consider the focus group research method since new phenomena regularly become of interest to IS researchers. For example, emerging technologies offer many topics: what are the new rules of social interactions with the increased use of social networking? Or, what are the potential alterations to group culture when individuals interact substantially in virtual worlds? In analyzing the list of “sole method – exploratory” papers in Table 1, it can be noted that most topics, with some exceptions, attempt to identify usage or managerial issues related to technology, systems, and IT management. In fact, several studies investigate how individuals use email, mobile technology, dashboards, CRM, health information systems, RFID, knowledge management, databases, application portfolio management, total cost of ownership, or digital libraries. In each of these studies, the main purpose of the focus group is to identify usage levels, determinants of use, or ways in which the system/technology/approach is used. As will be argued below, these exploratory studies are important in the research process towards future theory development. The second main group of studies investigate IS managers’ perceptions of various concepts related to IT management, such as IT staffing issues, IS-business alignment, teaching-research alignment, IS delivery options, IS careers, and general perceptions of IS 1 . These studies often use one large session where several 1 It should be noted that several articles in the sole method – exploratory category are from the “Developments in Practice” series published in the Communications of the AIS journal by McKeen and Smith (and other colleagues) McKeen, J. D., and Smith, H. "Developments in Practice Xxvii: Delivery It Functions: A Decision Framework," Communications of the Association for Information Systems (19) 2007, p 9, McKeen, J. D., and Smith, H. "Developments in Practice Xxxiv: Application Portfolio Management," Communications of the Association for Information Systems (26) 2010a, p 87, McKeen, J. D., and Smith, H. "Developments in Practice Xxxvii: Total Cost of Ownership," Communications of the Association for Information Systems (27) 2010b, p 627, McKeen, J. D., Smith, H., and Jin, J. "Developments in Practice Xxxii: Successful Strategies for It Staffing," ibid. (24:1) 2009, p 32, McKeen, J. D., and Smith, H. A. "Developments in Practice Xxiii: Creating and Evolving a Technology Roadmap," Communications of AIS (2006:18) 2006, pp. 451-463, McKeen, J. D., Smith, H. A., and Singh, S. "Developments in Practice Xx: Digital Dashboards: Keep Your Eyes on the Road," Communications of the Association for Information Systems (16) 2005, p 1, Smith, H. A., and McKeen, J. D. "Developments in Practice Ix: The Evolution of the Km Function," Communications of AIS (2003:12) 2003, pp. 69-79, ibid., Smith, H. A., and McKeen, J. D. "Developments in Practice Xiv: Marketing Km to the Organization," Communications of the Association for Information Systems (14) 2004, p 1, Smith, H. A., and McKeen, J. D. "Developments in Practice Xxiv: Information Management: The Nexus of Business and It," Communications of AIS (2007:19) 2007, pp. 34-46, Smith, H. A., McKeen, J. D., and Singh, S. "Developments in Practice Xxviii: Managing Perceptions of Is," ibid.:20), pp. 760-773.. Australasian Journal of Information Systems Volume 17 Number 2 2012 120 managers share their ideas on a general topic. This does not follow the typical rules of focus groups (4 to 6 participants; several focus groups to allow for comparisons), and as a result should be considered mainly as informative. However, there could still be value in these as idea generation for research questions, as will be discussed below. Sole Method Focus Groups for IS Theorizing Because the focus group research method allows concepts that would only emerge through interactions among individuals to be identified (Grant and Fitzgerald 2005; Hartman 2004), it is especially useful in IS theorizing when researchers are interested in group level phenomena (e.g., group cohesion in software development teams) or in phenomena that exist in the relationships among people (e.g., team satisfaction with technology used in teams within telecommuting environments). Sopreperez’ (2008) study of a UK Fire and Rescue service using focus groups highlights the value of group interaction. The researcher concludes, “some details only emerged when those in attendance were given the opportunity to discuss the incident as a group. Individual interviews would not have given this opportunity and observations would not have revealed attitudes and beliefs.” (p. 187). In information systems research, LeRouge and Niederman (2006) provide an example of how this type of focus groups can be useful in their study of architectural requirements for a public health knowledge system. They highlight the benefits of the focus group research method for identifying requirements for public health knowledge management systems: 1. “…stakeholder perspectives of the scope and potential of knowledge management systems were limited as the study began. By focusing on conceptual issues, the focus group method highlighted individual stakeholders’ reflections in building these conceptual insights… 2. … requirements gathering of systems such as knowledge bases may involve complex or vague topics as stakeholders look at new technologies or multifaceted constructs. When soliciting data from the field, it is sometimes difficult to achieve full explanation and understanding in surveys, especially in initial stages of research. Focus groups allow the gaps to be filled in by fellow participants. Limited moderator prompts may help ensure the topics and issues are understood, yet avoid excessive intervention by project directors, sponsors, or researchers… 3. …because the focus group draws participants from the full set of involved stakeholders, and because the participants are exposed to a wide range of ideas regarding the overall knowledge management program, these participants are themselves positioned to more easily continue diffusing knowledge about the program to nonparticipating stakeholders....” (pp. 18-19). The sole method exploratory focus groups can also be useful in generating research questions since the method allows researchers to probe subjects further when interesting concepts emerge from the “discussion” (Edmunds 1999) and allows concepts unclear to participants at the beginning of a session to become clarified through their interactions (Grant and Fitzgerald 2005; Hartman 2004). These would not result in analyzing theories as defined by Gregor, but can serve to identify research questions that are driven by the views and needs of participants for future work. In Table 1, this category is by far the largest, but as previously stated, one has to account for the “Developments in Practice” series by McKeen and Smith published in Communications of the AIS. Many phenomena of interest to information systems researchers could fall in the categories of exploratory concepts (e.g., understanding usage of emerging technologies, exploring new concerns of CIOs, etc.) or concepts initially unclear to participants (e.g., critical success factors of new IT strategies). For example, asynchronous but moderated focus groups could be used to elicit knowledge from domain experts. Building on market research where focus groups are used extensively to assess and provide feedback on products, services, or concepts, some IS researchers have explored the use of the Internet to conduct focus groups for eliciting information from groups of consumers online (Clapper and Massey 1996; Markham et al. 2006; Montoya-Weiss et al. 1998). Others have developed an Internet tool to Australasian Journal of Information Systems Volume 17 Number 2 2012 121 run online focus groups, which are said to reduce costs, enable participation of more dispersed subjects, and offer anonymity (Montoya-Weiss et al. 1998). As discussed by the authors, technology can help mitigate some of the limitations of face-to-face focus groups. Another potential use of this category of exploratory focus group studies is for requirements determination and design science (Leifer et al. 1994; O'hEocha et al. 2010; Peffers and Tuunanen 2005; Tremblay et al. 2010). Potential Issues with Using Single Method Focus Groups for IS Theorizing While the focus group research method as sole method – exploratory can be valuable in IS theorizing, there are some important pitfalls, many of which are found in the studies presented in Table 1. In particular, one has to be concerned with the lack of rigor regarding the sampling and research designs for this category of focus group research in the IS literature. The first issue is that several studies fail to disclose necessary information to evaluate the validity of the findings, such as the number of focus groups conducted, the number of individuals per focus group, or the nature of participants. All future IS studies using focus groups should clearly identify these important research design elements. Without proper information on participants, it is impossible to discuss the generalizability of the findings, an issue further discussed in the “Limitations of Focus Groups” section presented later in this paper. A second major issue is that many of the focus group-based studies in Table 1 (as single method) use only one focus group. As a result, these studies do not allow for comparisons across groups. While this may not be an issue for exploratory studies seeking to simply elicit information (generating lists of concepts), it is of major concern when one tries to develop a theory based on the one focus group. Depending on the composition of the group (homogenous or heterogeneous strangers), the researcher may find completely different findings. For example, a researcher interested in the success of IT implementations in a firm can assemble participants for a focus group A with all participants from the IT department, and arrive at a particular conclusion based on their inputs. The researcher could also assemble for focus group B with only participants from one user department, and will likely obtain different conclusions. Finally, the researcher can create a focus group C, with a mix of IT and non-IT participants. 2 Each of the groups would provide a different perspective on the success of the IT implementation in the firm. Each perspective could be valuable, depending on the interests or research question of the researcher. If the researcher wants to see how biased the view of the IT team is, then using groups A and B would provide valuable insights (using more than one group of each to achieve saturation). An example from Table 1 where this approach was well done is the study by Reid et al (2010) who purposely designed the study with several same sex focus groups to identify the challenges women face in the IS workplace were perceived similarly or differently by men and women. The use of several homogenous groups allowed the researchers to “validate” their findings across groups. In summary, the research design should identify which population is of interest to the particular research question, and then include enough focus groups and participants to achieve saturation, i.e., enough members from all stakeholder groups, relevant to the question and the intended references (what are we trying to say that can be obtained from the focus group findings?). Other researchers suggest that focus groups should only be used when part of a multi-method methodology. For example, in discussing the nexus between teaching and research, Grant and Fitzgerald (2005) conclude that “the issue of measurement bias is a major problem for users of focus groups” (p. 50) and that “focus groups are not beneficial as standalone research methods” (p. 52). It 2 An anonymous reviewer is to be thanked for inspiring this example. Australasian Journal of Information Systems Volume 17 Number 2 2012 122 should be noted, however, that some of the decisions involving the design of their focus groups could have impacted their limited findings. First, the authors ran only two focus groups, not seeking saturation in their findings (Carlsen and Glenton 2011). More importantly, the authors did not select homogeneous strangers (subjects relatively “compatible” with each other) for their focus groups. For example, their second focus group included senior faculty, including a head of a school and a professor with an established research program, two faculty with established research but one new to the department, and two non-research active faculty, both long-standing and one responsible for the largest undergraduate program. Academics active on committees in various academic institutions worldwide would probably recognize the potential issues of power plays, coalitions, and the like in such a group of individuals. It is therefore possible that the lack of “appropriate” results identified with the focus groups could be the result of the design of the study itself as opposed to weaknesses in the method. The authors suggest that other researchers may have had biased findings when using focus groups because of the “power and political interplay being exercised to give the illusion of a consensus.” However, as previously stated, it is this interplay that provides strength to focus group findings in theory development related to IS phenomena where conducting individual interviews or surveys may not reveal important constructs. The important implication for IS theorizing is that researchers must recognize whether the concepts of interest are meant to emerge from group discussions or are embedded in the social context of groups or communities (e.g., satisfaction with technology used in teams within telecommuting environments); or, whether they are clearly of individual-only concern (e.g., perceptions of mobile complexity). For example, a study that requires an understanding of individual perceptions or attitudes on their own, not embedded in social interactions, would benefit more from individual interviews or quantitative methods (e.g. an understanding of self-efficacy). In summary, and as suggested by Morgan (1997), the focus group research method as sole method can provide greater insights than the sum of individual interviews through its synergistic effects. This is consistent with the advantages of the focus group research method presented by LeRouge and Niederman (2006), which could apply to other IS studies when concepts would only emerge through interactions among individuals (e.g., cultural effects in software development team interactions). In other words, when researchers are interested in phenomena where interactions of individuals are important, the focus group research method can be a better research method than individual interviews. If, however, the researcher is interested in analyzing individuals’ intra-personal cognitions or perceptions (e.g., self-efficacy), the interview research method (or other methods like surveys) might be more useful (although focus groups could still be used in the early stages of the research to elicit dimensions). FOCUS GROUP AS SOLE METHOD - EXPLANATORY There was only one study where the focus group research method was used as sole method to provide explanations (test) of concepts. In this study, Sutton et al (2008) conduct five focus groups with various stakeholders representing internal constituency groups (IS security, internal IT audit, and e- commerce development managers) and external constituency groups (ecommerce consultants, and external IT auditors) to identify critical risk factors to assess the impact of B2B e-commerce on overall enterprise risk. While the main effort is in the elicitation of the risk factors, the authors go beyond simple enumeration and quantification of concepts to testing the differences in the concepts identified between constituency groups, thereby providing a test of their findings. Many studies identified in the sole method category could have taken these further steps to compare results across groups. For example, the authors could have conducted between group analyses for their data across countries in Jarvenpaa and Lang (2005) and Jarvenpaa et al (2003), or between groups of administrators, educators, and students in LeRouge and Niederman (2006). This does not take away Australasian Journal of Information Systems Volume 17 Number 2 2012 123 from what has been accomplished in terms of knowledge elicitation in these studies since the goal in focus group research is usually saturation, where repeated focus group sessions should result in similar findings to a point where no new insight is gained from additional focus groups (Carlsen and Glenton 2011). Group comparisons in this case would not provide any differences. Yet, the possibility of comparing findings across diverse groups can provide further insights into the exploratory findings previously obtained (assuming saturation is reached with homogenous groups first and that there is availability of different stakeholder groups). FOCUS GROUP AS PART OF MULTI-METHOD: EXPLORATORY Published journal articles using focus groups that fall in the multi-method category can also be exploratory or explanatory. Exploratory papers tend to combine findings from focus groups with surveys and/or interview data to obtain a more global picture of the phenomenon of interest. In fact, findings from Table 1 suggest that the most often used complementary research method to focus groups is interviews (found in 10 of the multi-method exploratory studies) and surveys (found in nine of the multi-method exploratory studies). Other methods used in combination with focus groups include observations (3), logs (2), secondary data (2), and an expert evaluation (van Velsen et al. 2008). An analysis of the studies in this category in Table 1 reveals some interesting patterns, namely that the multi-method exploratory research studies explore similar topics as the single method exploratory focus groups, and suffer from some of the same issues. As can be seen from the Table, the main topics continue to be on usage (e.g., of analytical Question-Answering systems, digital libraries, ERP, database marketing, cellular technology, RFID) or identification (e.g., behaviours in virtual worlds, evaluation factors, IT transfers, competencies, resistance factors, trust factors). More importantly, there remain issues of lack of information with four of the 16 studies not specifying the number of focus groups and several studies not indicating the number of participants or the nature of the participants. Three of the studies were also conducted with only one focus group. While it might be less of a concern since the focus group is usually used only for eliciting concepts, it remains that using only one focus group might result in some missing concepts, as explained previously. Multi-Method (Exploratory) Focus Groups for IS Theorizing The use of multi-method exploratory focus groups is consistent with the case study method to theory building (Eisenhardt 1989; Orlikowski and Baroudi 1991; Strauss and Corbin 1990; Yin 1994) where multiple sources of data must be used (Myers 1997; Yin 1994). A good example of this type of study is the work of Manning (1996) who spent considerable time and effort studying the use of technologies by police officers. Combining individual interviews with focus group data and observations allowed the researcher to gain a deep understanding of such use. There was no stated research model to be tested in the paper, and no particular grounded theory was presented as a result of the analyses in the paper. Yet, the findings provide for an in-depth discussion of organizational use of technology, and the utility of the dramaturgical perspective used by the author for the analysis of technological innovation in organizational contexts. As used in later studies, the technology-in- practice perspective suggests that rules regulating technology use are developed through recurrent use of the technology (Orlikowski 2000). Eisenhardt (1989) proposes several steps in the process of theory building: 1. getting started (defining the research question and any a priori constructs), 2. selecting cases (not having hypotheses but selecting a specified population), 3. crafting instruments and protocols (making sure that multiple data collection methods are used, both qualitative and quantitative, and using multiple investigators), 4. entering the field (where data collection and analysis overlap, and data collection can be flexible and opportunistic), 5. analyzing data (both within case and cross-case), 6. shaping hypotheses (with iterative tabulation of evidence, using replication and/or logic across cases; searching for evidence for Australasian Journal of Information Systems Volume 17 Number 2 2012 124 why behind the relationships identified), 7. enfolding literature (comparing with conflicting and similar literature), and 8. reaching closure (with theoretical saturation when possible). While her work was more specifically targeted at case studies, the concepts to the multi-method focus group research can be applied as follows. Step 1 remains the definition of the problem or research question of interest. Step 2 involves selecting the appropriate population for the focus groups that is relevant to the research question. Step 3 involves developing the focus group protocol (see Appendix A for more details) and selecting the complementary research methods. Step 4 involves conducting the focus groups while allowing new insights to be incorporated into subsequent focus groups or other data collection methods (like interviews). Steps 5 to 8 remain similar (analysing the collected data, shaping hypotheses, comparing to the literature, and reaching closure), with the objective of developing theory. An example of the use of focus groups for grounded theory work comes from the author’s (and colleagues) study of online children privacy where the researchers were trying to identify what factors would make parents use an online privacy protection tool (Bélanger et al. 2009). Most researchers would use long-standing technology adoption models as theoretical foundations for their work. Using the Technology Acceptance Model (Davis 1989) or the Unified Theory of Acceptance and Use of Technology (Venkatesh et al. 2003), researchers would likely create a model with perceived usefulness (or performance expectancy), perceived ease of use (or effort expectancy), and a variety of other factors such as social influence or facilitating conditions, as impacting likely intentions to use the software. Instead, we conducted focus groups and interviews with actual parents to build an emergent model, which did result in the inclusion of ease of use and usefulness, but also awareness of laws and risks, trust, and behavioural control, among others. The focus groups allowed parents to discuss what was important to them, and to build on one another’s comments. In the real world (which they were in), parents often discuss with one another the best ways to protect children, and this emerged from the focus groups. More importantly, we found that intentions to use the privacy protection tool was not the ultimate dependent variable, but that various techniques (such as keeping a home computer in view) were as important if not more important for parents in protecting their children’s online privacy. In this case, parents were considered experts since their level of knowledge was part of the information the study was seeking. The grounded theory model derived from the focus group data was then tested using a large scale survey. In summary, while the findings from Table 1 reveal that several issues remain in many of the multi- method exploratory focus group research studies in terms of sampling, the examples show that the exploratory focus group used in conjunction with other research methods can be useful in developing grounded IS theories. FOCUS GROUP AS PART OF MULTI-METHOD: EXPLANATORY The second type of multi-method focus group research papers in information systems involve some form of testing using one of the “other” methods besides the focus group research method, providing a positivist view of research. Most of them (10) from the sampled papers in Table 1 use the focus group research method to generate constructs or items for model or scale development, and then use survey questionnaires (and sometimes interviews) for model testing. A few others (4) use focus groups after survey data have been collected and analysed to provide further insights into the findings, or to validate developed frameworks. Only one paper uses focus group data to test hypotheses (Xu et al. 2011). Using focus groups for concept generation followed by testing is consistent with the research cycle of exploratory to explanatory studies since the use of focus groups as part of a larger research program offers triangulation for validating research, as called for by McGrath (1982). Australasian Journal of Information Systems Volume 17 Number 2 2012 125 In Table 1, it can be seen that research papers in the explanatory multi-method focus group research category differ from the exploratory papers in many ways. First, topics of interest move beyond usage and identification to explanation of a variety of IS phenomena. For example, there are studies of impacts (e.g., job characteristics on IS employee satisfaction (Galup et al. 2008)) or adoption (e.g., Push-to-Talk technologies (Dickinger et al. 2008)), or development of new IS constructs such as Concern for Information Privacy (Smith et al. 1996) or IS development project complexity (Weidong and Lee 2005). A second major difference is that most of the studies provide detailed information on the focus group sample, quantity, and the nature of participants, and most of them include many focus groups (only one study uses one focus group). The fact that more planning appears to have been done for those studies as opposed to the previously discussed ones can be due to 1. the more advanced state of knowledge of the topics of interest (as opposed to exploratory topics), or 2. the fact that many of these studies are published in leading IS journals, which might be more demanding regarding details to be provided. Multi-Method (Explanatory) Focus Groups for IS Theorizing In analysing the papers in Table 1, it is clear that the multi-method explanatory focus group research method has been used for theory development, more specifically for the “Explaining and Predicting” types of theories proposed by Gregor (2006). For these theories, where researchers seek to explain what is occurring and provide testable predictions with causal explanations, focus groups can be used to elicit items for further testing and/or to develop the testable model (similar to what was done with exploratory focus groups presented before). This is most beneficial when IS researchers are interested in concepts that would only emerge through interactions among individuals. In these cases, the focus group research method can provide greater insights than the sum of individual interviews through its synergistic effects, allowing researchers to obtain a more realistic emergent model for later testing. If these interactions are important to the concepts of interest, a model built from interview data only might be missing some key constructs or relationships. A good example of using focus groups in developing explaining and predicting theories (Gregor 2006) is the work of Dickinger et al. (2008) who employ the focus group research method to identify factors that affect user intentions to use Push to Talk Technology (PTT). More specifically, their methodology combines literature review and 10 interviews to define an initial model, followed by three focus groups to develop initial hypotheses to be tested with survey data. As shown in Figure 1, their use of multiple methods follows a clear evolution from exploratory methodologies (literature, interviews and focus groups) to confirmative (explanatory) methodologies (survey and analyses). Australasian Journal of Information Systems Volume 17 Number 2 2012 126 Literature Review Expert Interviews Focus Groups Survey Analysis & Presentation Determination of Key Success Factors Definition of Acceptance Model Formulation of Hypotheses Empirical testing Presentation of Results E x p lo ra ti v e R e s e a rc h P ro c e s s C o n fi rm a ti v e R e s e a rc h P ro c e s s Figure 1. Research Process to Study Adoption of Push to Talk Technology (from Dickinger et al. 2008) Another example from a study of attraction and retention of minorities in computing majors conducted by the author and colleagues (Lewis et al. 2007) highlights the value of focus groups in unearthing key concepts. In the study, the researchers conducted a survey based on the literature to explain why minorities stay into or leave computing majors. The survey included a number of intra- personal factors identified from the literature, such as coping styles, ethnic identity, affinity with the major, and even emotional intelligence. Some findings from the survey data were surprising, so subsequent to conducting the survey a series of seven focus groups were conducted with minority computing students. The focus group findings confirmed some of the previously identified factors, such as coping strategies, and helped identify additional factors such as stressors. Interestingly, the visual data helped identify a number of additional factors related to stereotypes in the major that helped explain prior findings. For example, many of the male students reacted with rolled eyes when the female member of one focus group discussed the issue of unbalanced life; or body reactions indicated that even students who denied that lack of confidence was a major stressor, actually thought it was important (but would never admit it). These findings were very helpful to the researchers in better understanding the retention issues they were studying. MULTI-METHOD: OTHERS Finally, there were two studies in the sample of papers that were classified as “others” because they combined focus groups with other methods in non-standard ways. In one of the “other” studies, Smith and colleagues (2010) use 22 focus groups (as well as interview, forums and discussions) as interventions between a series of online surveys to study information systems security standard compliance in government. In this case, focus groups are viewed as action research. In the other study, Chang Lee and Kwon (2008) used a focus group to extract the cognitive maps of experts and Australasian Journal of Information Systems Volume 17 Number 2 2012 127 create a collective cognitive map to be used in the development of a decision support system using avatars. Because the interest of the researchers was on defining the collective cognitive map, the use of focus groups was very appropriate. In fact, it is likely that other Group Support System research use focus groups to create cognitive mapping systems. This type of research would be considered the development of a design and action theory, an area that many researchers suggest need further work (Bélanger and Crossler 2011; Gregor 2006; Hevner et al. 2004). For this type of theory building, focus groups can be used prior to the IT artefact design as knowledge elicitation (as done in the above mentioned study) or after the IT artefact is designed for testing of the artefact (e.g., usability testing). LIMITATIONS OF THE FOCUS GROUP RESEARCH METHOD The focus group research method is not without its limitations. A good summary of the benefits and limitations of focus group research is provided by Morgan and Spanish (1984): “… the strengths of focus groups come from a compromise between the strengths found in other qualitative methods. Like participant observation, they allow access to a process that qualitative researchers are often centrally interested in: interaction. Like in-depth interviewing, they allow access to the content that we are often interested in: the attitudes and experiences of our informants. As a compromise, focus groups are neither as strong as participant observation on the naturalistic observation of interaction, nor as strong as interviewing on the direct probing of informant knowledge, but they do a better job of combining these two goals than either of the other two techniques." (p. 260). Some criticisms of the focus group research methods are that they are too dependent on the skills of the moderator, they have an unclear level of analysis, and have measurement bias (Grant and Fitzgerald 2005). Others have stated how focus groups represent collective thinking only; however, the main criticism is often validity. Having conducted several research projects with focus groups (as part of large research programs), I know that reviewers often question whether the findings of a study that uses the focus group method are valid. Clearly, positivist quantitative approaches to research have the favour of many researchers because of their ability to offer statistical validity (Cook and Campbell 1979) and to be “verifiable” through future studies (in reality, as a percentage, few published IS studies have actually been replicated in research). Although one can obtain quantifiable results using focus groups (as with interviews), focus groups are not designed for statistical validity (or internal validity as described below). Yet, they do meet other validity objectives as identified by McGrath (1982). The quote at the beginning of this paper is representative of the issues faced by researchers when making research design choices. McGrath (1982) proposes that there are several types of validity that must be balanced: internal validity, external validity, and relevance. McGrath also suggests that these validities represent mutually conflicting objectives: “There is no way […] to maximize all three conflicting desiderata of the research strategy domain.” Table 3 summarizes how focus groups can address these three validities. In terms of generalizability, focus groups deal with a selected subset of a population, which is typically not randomly selected. In conducting research with non-probability sampling, whether qualitative or quantitative, researchers must clearly define the populations to which the findings apply (Seddon and Scheepers 2011). In discussing case studies, Yin (1994) suggests there are two types of generalizability: statistical and analytical, and argues that for qualitative research analytical generalizability is necessary. Analytical generalizability occurs when the results of the study provide an analytical understanding of individuals’ behaviours or motivations. When focus groups are properly conducted and documented, they provide an understanding of why individuals think or behave in a certain way, as can be seen from the examples presented in the paper. Furthermore, since Australasian Journal of Information Systems Volume 17 Number 2 2012 128 subjects are considered experts in the topic of interest, their collective voice is representative of what others like them think of the subject. As discussed in Lee and Baskerville (2003), qualitative research can provide claims of generalizability when the context of the study and the boundaries are clearly delineated. In their recent discussion on generalizability, Seddon and Scheepers (2011) highlight the need for qualitative studies (or as relevant to this discussion, focus groups as sole method or as part of a multi-method effort) to clearly disclose and discuss the boundary conditions to which their findings apply. If researchers do provide their protocol as part of their publications, and clearly describe the population of interest, other researchers should be able to replicate the findings, just as survey instruments can be validated in multiple studies. Unfortunately, as demonstrated by studies on generalizability (Lee and Baskerville 2003; Seddon and Scheepers 2011), and consistent with the findings in Table 1, researchers often fail to disclose this important information. Validity Description Focus Group Research Requirement External validity (generalizability) Extent to which the presumed finding can be generalized to and across different types of persons, settings, and times (populations). Focus groups provide analytical generalizability, which is an analytical understanding of individuals’ behaviors or motivations. Internal validity Precision in the control and measurement of variables. Focus group researchers need to provide a thorough description of the complete focus group process and data interpretation. Relevance (realism) Whether the phenomenon of concern is studied in a realistic setting. Focus group participants are considered experts for the particular domain of interest. Table 3. Validity and Focus Groups In terms of internal validity, or reliability, it is clear that this cannot be accomplished through controls and precision of the measurement method and its ability to reproduce the same results repeatedly. However, Sykes (1991) discusses how in qualitative research, reliability refers to the role of the researcher in the data collection and in her interpretation. She recommends the qualitative researcher provide a thorough description of the complete process used, including preparation, data collection, and analysis. In the context of focus groups, researchers should clearly indicate the research purpose, the selection criteria, and method used to solicit subjects, the focus group protocol, the capabilities and/or training of the moderator, the settings and tools used for the focus group sessions, the number of sessions held and the time spent at each session, and an actual description of the focus group sessions themselves (including a description of the transcriptions, analyses performed, etc.). Clearly, based on the findings from Table 1, IS researchers should be more careful in this regard when using the focus group research method for their studies. Finally, an additional potential limitation of the focus group research method is groupthink, a psychological phenomenon where individuals’ thinking tends to be overridden by the group’s Australasian Journal of Information Systems Volume 17 Number 2 2012 129 thinking as individuals seek harmony and reduced conflict in group conversations. Groupthink has been identified as an issue in other research methods such as the Delphi research method 3 . It is the role of the focus group moderator to avoid such a phenomenon from occurring when conducting the focus group sessions. CONCLUSION The focus group research method is a qualitative research method of interest to information systems researchers. While it has some limitations with respect to statistical validity, if properly planned, conducted, and documented, the focus group research method can offer valuable insights to IS researchers for a plethora of information systems phenomena where individuals are embedded in collectives (Morgeson and Hofmann 1999) that impact the emergence of their perceptions, attitudes and behaviors. More importantly, when IS researchers are interested in constructs that emerge from or exist in group interactions, when they are exploring new IS concepts to be tested later, or when they seek a deeper understanding of constructs in established theories, the focus group research method can provide valuable insights for IS theorizing. ACKNOWLEDGEMENTS The paper greatly benefited from comments from Dr. Steve Sheetz, Kathy Enget, as well as the editors and reviewers of AJIS. REFERENCES Baskerville, R. L., and Myers, M. D. "Fashion Waves in Information Systems Research and Practice," MIS Quarterly (33:4) 2009, pp. 647-662. Bélanger, F., and Crossler, R. E. "Privacy in the Digital Age: A Review of Information Privacy in Is Literature," MIS Quarterly (35:4), December 2011, pp. 1017-1041. Bélanger, F., Crossler, R. E., Hiller, J. S., Park, J.-M., and Hsiao, M. "Children Online Privacy: Issues with Parental Awareness and Control," in: Annals of Emerging Research in Information Assurance, Security and Privacy Services, H.R. Rao and S. Upadhyaya (eds.), Reed Elsevier, 2009, pp. 311-333. Bishop, A. P., Neumann, L. J., Star, S. L., and Merkel, C. "Digital Libraries: Situating Use in Changing Information Infrastructure," Journal of the American Society for Information Science and Technology (51:4) 2000, p 394. Bryman, A. Quantity and Quality in Social Research Unwin Hyman, London, 1988. Campbell, B., Kay, R., and Avison, D. "Strategic Alignment: A Practitioner's Perspective," Journal of Enterprise Information Management (18:6) 2005, pp. 653-664. Carlsen, B., and Glenton, C. "What About N? A Methodological Study of Sample-Size Reporting in Focus Group Studies," BMC medical research methodology (11:26) 2011, pp. 2-10. Chang Lee, K., and Kwon, S. "A Cognitive Map-Driven Avatar Design Recommendation Dss and Its Empirical Validity," Decision Support Systems (45:3) 2008, pp. 461-472. 3 A reviewer is to be thanked for this insightful addition to the limitations. Australasian Journal of Information Systems Volume 17 Number 2 2012 130 Chesney, T., Coyne, I., Logan, B., and Madden, N. "Griefing in Virtual Worlds: Causes, Casualties and Coping Strategies," Information Systems Journal (19:6) 2009, pp. 525-548. Clapper, D. L., and Massey, A. P. "Electronic Focus Groups: A Framework for Exploration," Information & Management (30:1) 1996, p 43. Cook, T. D., and Campbell, D. T. Quasi-Experimental Design: Design and Analysis Issues for Field Settings Rand McNally, Skokie, IL, 1979, pp. 223-336. Cooper, G., and Heinze, A. "Centralisation of Assessment: Meeting the Challenges of Multi-Year Team Projects in Information Systems Education," Journal of Information Systems Education (18:3), Fall2007 2007, pp. 345-355. Davis, F. D. "Perceived Usefulness, Perceived Ease of Use and User Acceptance of Information Technology," MIS Quarterly (13:3) 1989, pp. 319-340. Dickinger, A., Arami, M., and Meyer, D. "The Role of Perceived Enjoyment and Social Norm in the Adoption of Technology with Network Externalities," European Journal of Information Systems (17:1) 2008, pp. 4-11. Dunning, K. A., Vijayaraman, B. S., Krovi, R., and Kahai, P. S. "Graduate E-Business Program Design and Evaluation," Journal of Computer Information Systems (42:1), Fall2001 2001, p 58. Dwyer, C., and Knapp, C. A. "How Useful Is Is 2002? A Case Study Applying the Model Curriculum," Journal of Information Systems Education (15:4) 2004, pp. 409-416. Edmunds, H. The Focus Group Research Handbook NTC Contemporary, Lincolnwood, IL, 1999. Eisenhardt, K. M. "Building Theory from Case Study Research," Academy of Management Review (14:4) 1989, pp. 532-550. Fern, E. F. "The Use of Focus Groups for Idea Generation: The Effects of Group Size, Acquaintanceship, and Moderator on Response Quantity and Quality," Journal of Marketing Research (19:1) 1982, pp. 1-13. Fern, E. F. Advanced Focus Group Research Sage Publications, Thousand Oaks, CA, 2001. Fitch, D. "Structural Equation Modeling the Use of a Risk Assessment Instrument in Child Protective Services," Decision Support Systems (42:4) 2007, pp. 2137-2152. Galup, S. D., Klein, G., and Jiang, J. J. "The Impacts of Job Characteristics on Is Employee Satisfaction: A Comparison between Permanent and Temporary Employees," Journal of Computer Information Systems (48:4), Summer2008 2008, pp. 58-68. Glass, R. "Making Research More Relevant While Not Diminishing Its Rigor," IEEE Software (26:2) 2009, p 96. Goh, D., Ang, R., and Lee, C. "Determining Services for the Mobile Tourist," The Journal of Computer Information Systems (51:1) 2010, pp. 31-40. Grant, K., and Fitzgerald, S. "The Nexus between Teaching and Research: A Qualitative Study Using Two Focus Group on Academic Information Systems Teachers," Electronic Journal of Business Research Methods (3:1) 2005, pp. 37-56. Gregor, S. "The Nature of Theory in Information Systems," MIS Quarterly (30:3), September 2006, pp. 611-642. Hartman, J. "Using Focus Groups to Conduct Business Communciations Research," The Journal of Business Communications (41:4) 2004, pp. 402-410. Hevner, A. R., March, S. T., Park, J., and Ram, S. "Design Science in Information Systems Research," MIS Quarterly (28:1) 2004, pp. 75-105. Hill, C. E., Loch, K. D., Straub, D. W., and El-Sheshai, K. "A Qualitative Assessment of Arab Culture and Information Technology Transfer," Journal of Global Information Management (6:3) 1998, pp. 29-38. Huang, E., and Lin, S. "Do Knowledge Workers Use E-Mail Wisely?," The Journal of Computer Information Systems (50:1) 2009, pp. 65-73. Australasian Journal of Information Systems Volume 17 Number 2 2012 131 Jarvenpaa, S. L., and Lang, K. R. "Managing the Paradoxes of Mobile Technology," Information Systems Management (22:4), Fall2005 2005, pp. 7-23. Jarvenpaa, S. L., Lang, K. R., Takeda, Y., and Tuunainen, V. K. "Mobile Commerce at Crossroads," Communications of the ACM (46:12) 2003, pp. 41-44. Jones, M., Zmud, R., and Clark, T., Jr. "Erp in Practice: A Snapshot of Post-Installation Perception and Behaviors," Communications of the Association for Information Systems (23:Article 25) 2008, pp. 437-462. Kelly, D., Wacholder, N., Rittman, R., Sun, Y., Kantor, P., Small, S., and Strzalkowski, T. "Using Interview Data to Identify Evaluation Criteria for Interactive, Analytical Question- Answering Systems," Journal of the American Society for Information Science & Technology (58:7) 2007, pp. 1032-1043. Kesner, R. M. "Business School Undergraduate Information Management Competencies: A Study of Employer Expectations and Associated Curricular Recommendations," Communications of AIS (2008:23) 2008, pp. 633-654. Khalifa, M., and Limayem, M. "Drivers of Internet Shopping," Communications of the ACM (46:12) 2003, pp. 233-239. Klaus, T., and Blanton, J. "User Resistance Determinants and the Psychological Contract in Enterprise System Implementations," European Journal of Information Systems (19:6) 2010, pp. 625-636. Krasnova, H., Spiekermann, S., Koroleva, K., and Hildebrand, T. "Online Social Networks: Why We Disclose," Journal of Information Technology (Palgrave Macmillan) (25:2) 2010, pp. 109- 125. Krueger, R. A. Focus Groups: A Practical Guide for Applied Research Sage Publications, Thousand Oaks, CA, 1994. Krueger, R. A., and Casey, M. A. Focus Groups: A Practical Guide for Applied Research Sage, Thousand Oaks, CA, 2000. Le Rouge, C., and Niederman, F. "Information Systems and Health Care Xi: Public Health Knowledge Management Architecture Design: A Case Study," Communications of AIS (2006:18) 2006, pp. 2-54. Lee, A. S., and Baskerville, R. L. "Generalizing Generalizability in Information Systems Research," Information Systems Research (14:3) 2003, pp. 221-243. Lee, D. M. S., Trauth, E. M., and Farwell, D. "Critical Skills and Knowledge Requirements of Is Professionals: A Joint Academic/Industry Investigation," MIS Quarterly (19:3) 1995, pp. 313-340. Lehaney, B., Clarke, S., Kimberlee, V., and Spencer-Matthews, S. "The Human Side of Information Development: A Case of an Intervention at a British Visitor Attraction," Journal of End User Computing (11:4) 1999, pp. 33-39. Leifer, R., Lee, S., and Durgee, J. "Deep Structures: Real Information Requirements Determination," Information & Management (27:5) 1994, pp. 275-285. Lewis, T., Smith, W. J., Bélanger, F., and Harrington, K. V. "A Qualitative Analysis of Computing Students’ Professional Identity and Its Relationship to Strategies for Coping with Stressors in the Computing Disciplines," America’s Conference on Information Systems (AMCIS), Keystone, Colorado, 2007. Maheshwari, S., Pierce, A., and Zapatero, E. "Understanding the Lack of Minority Representation in Graduate Programs in Computer Science and Information Technology: A Focus Group Study of Student Perceptions," Academy of Information and Management Sciences Journal (12:1/2) 2009, pp. 71-90. Manning, P. K. "Information Technology in the Police Context: The "Sailor" Phone," Information Systems Research (7:1) 1996, pp. 52-62. Australasian Journal of Information Systems Volume 17 Number 2 2012 132 Markham, S., Gatlin-Watts, R., and Cangelosi, J. "The Internet and Its Relationship to Buyer Behavior Theory," Journal of Internet Commerce (5:1) 2006, pp. 69-86. Martz, B., Zhang, X., and Ozanich, G. "Information Systems and Healthcare Xix: Developing an Integrative Health Informatics Graduate Curriculum," Communications of AIS (2007:19) 2007, pp. 645-654. McGrath "Dilemmatics: The Study of Research Choices and Dilemmas," in: Judgment Calls in Research J.E. McGrath, J. Martin and R.A. Kulka (eds.), Sage, Newbury Park, CA, 1982, pp. 69-102. McKeen, J. D., and Smith, H. "Developments in Practice Xxvii: Delivery It Functions: A Decision Framework," Communications of the Association for Information Systems (19) 2007, p 9. McKeen, J. D., and Smith, H. "Developments in Practice Xxxiv: Application Portfolio Management," Communications of the Association for Information Systems (26) 2010a, p 87. McKeen, J. D., and Smith, H. "Developments in Practice Xxxvii: Total Cost of Ownership," Communications of the Association for Information Systems (27) 2010b, p 627. McKeen, J. D., Smith, H., and Jin, J. "Developments in Practice Xxxii: Successful Strategies for It Staffing," Communications of the Association for Information Systems (24:1) 2009, p 32. McKeen, J. D., and Smith, H. A. "Developments in Practice Xxiii: Creating and Evolving a Technology Roadmap," Communications of AIS (2006:18) 2006, pp. 451-463. McKeen, J. D., Smith, H. A., and Singh, S. "Developments in Practice Xx: Digital Dashboards: Keep Your Eyes on the Road," Communications of the Association for Information Systems (16) 2005, p 1. Miles, M. B., and Huberman, A. M. Qualitative Data Analysis: An Expanded Sourcebook Sage Publications, Thousand Oaks, CA, 1994. Montoya-Weiss, M. M., Massey, A. P., and Qapper, D. L. "On-Line Focus Groups: Conceptual Issues and a Research Tool," European Journal of Marketing (32:7/8) 1998, p 713. Morgan, D. L. Focus Groups as Qualitative Research, (2 ed.) SAGE Thousand Oaks, CA, 1997, p. 80. Morgan, D. L., and Spanish, M. T. "Focus Groups: A New Tool for Qualitative Research," Qualitative Sociology (7:3) 1984, pp. 253-270. Morgeson, F. P., and Hofmann, D. A. "The Structure and Function of Collective Constructs: Implications for Multilevel Research and Theory Development," Academy of Management Review (24:2), Apr 1999, pp. 249-265. Myers, M. D. "Qualitative Research in Information Systems," in: MISQ Discovery, 1997. Nahar, N., Lyytinen, K., Huda, N., and Muravyov, S. V. "Success Factors for Information Technology Supported International Technology Transfer: Finding Expert Consensus," Information & Management (43:5) 2006, pp. 663-677. O'hEocha, C., Conboy, K., and Xiaofeng, W. "Using Focus Groups in Studies of Isd Team Behaviour," Electronic Journal of Business Research Methods (8:2) 2010, pp. 119-131. Orlikowski, W. J. "Using Technology and Constituting Structures: A Practice Lens for Studying Technology in Organizations," Organization Science (11), July-August 2000, pp. 404-428. Orlikowski, W. J., and Baroudi, J. J. "Studying Information Technology in Organizations: Research Approaches and Assumptions," Information Systems Research (2) 1991, pp. 1-28. Otondo, R. F., Pearson, A. W., Pearson, R. A., Shaw, J. C., and Shim, J. P. "Managerial Problem- Solving in the Adoption of Radio Frequency Identification Technologies," European Journal of Information Systems (18) 2009, pp. 553-569. Payton, F. C., and Zahay, D. "Understanding Why Marketing Does Not Use the Corporate Data Warehouse for Crm Applications," Journal of Database Marketing (10:4) 2003, pp. 315- 326. Australasian Journal of Information Systems Volume 17 Number 2 2012 133 Peffers, K., and Tuunanen, T. "Planning for Is Applications: A Practical, Information Theoretical Method and Case Study in Mobile Financial Services," Information & Management (42:3) 2005, pp. 483-501. Pettigrew, K. E., Durrance, J. C., and Unruh, K. T. "Facilitating Community Information Seeking Using the Internet: Findings from Three Public Library-Community Network Systems," Journal of the American Society for Information Science and Technology (53:11) 2002, pp. 894-903. Price, R., and Shanks, G. "A Semiotic Information Quality Framework: Development and Comparative Analysis," Journal of Information Technology (20:2) 2005, pp. 88-102. Raitoharju, R., Heiro, E., Kini, R., and D'Cruz, M. "Challenges of Multicultural Data Collection and Analysis: Experiences from the Health Information System Research," Electronic Journal of Business Research Methods (7:1) 2009, pp. 75-82. Ramesh, B., and Jarke, M. "Toward Reference Models for Requirements Traceability," IEEE Transactions on Software Engineering (27:1) 2001, pp. 58-93. Reid, M., Allen, M., Armstrong, D., and Riemenschneider, C. "Perspectives on Challenges Facing Women in Is: The Cognitive Gender Gap," European Journal of Information Systems (19:5) 2010, p 526. Resatsch, F., Sandner, U., Leimeister, J., and Krcmar, H. "Do Point of Sale Rfid-Based Information Services Make a Difference? Analyzing Consumer Perceptions for Designing Smart Product Information Services in Retail Business," Electronic Markets (18:3) 2008, p 216. Rosemann, M., and Vessey, I. "Toward Improving the Relevance of Information Systems Research to Practice: The Role of Applicability Checks," MIS Quarterly (32:1) 2008, pp. 1-22. Scott, C. K., Fuller, M. A., MacIndoe, K. M., and Joshi, K. D. "More Than a Bumper Sticker: The Factors Influencing Information Systems Career Choices," Communications of AIS (2009:24) 2009, pp. 7-26. Seddon, P. B., and Scheepers, R. "Towards the Improved Treatment of Generalization of Knowledge Claims in Is Research: Drawing General Conclusions from Samples," European Journal of Information Systems ) 2011, pp. 1-16. Shuk Ying, H., Pan, G., and Ferguson, C. "The Information Systems-Accounting Nexus: Lessons from an Australian Institution," Communications of AIS (2008:22) 2008, pp. 197-210. Smith, H., and McKeen, J. "Developments in Practice Xxxv: Building a Strong Relationship with the Business," Communications of the Association for Information Systems (26) 2010a, p 91. Smith, H., and McKeen, J. "Developments in Practice Xxxvi: How to Talk So Business Will Listen ... And Listen So Business Will Talk," Communications of the Association for Information Systems (27:-) 2010b, p 207. Smith, H., and McKeen, J. "Enabling Collaboration with It," Communications of the Association for Information Systems (28) 2011, p 1. Smith, H. A., and McKeen, J. D. "Developments in Practice Ix: The Evolution of the Km Function," Communications of AIS (2003:12) 2003, pp. 69-79. Smith, H. A., and McKeen, J. D. "Developments in Practice Xiv: Marketing Km to the Organization," Communications of the Association for Information Systems (14) 2004, p 1. Smith, H. A., and McKeen, J. D. "Developments in Practice Xxiv: Information Management: The Nexus of Business and It," Communications of AIS (2007:19) 2007, pp. 34-46. Smith, H. A., McKeen, J. D., and Singh, S. "Developments in Practice Xxviii: Managing Perceptions of Is," Communications of AIS (2007:20) 2007, pp. 760-773. Smith, H. J., Milberg, S. J., and Burke, S. J. "Information Privacy: Measuring Individuals' Concerns About Organizational Practices," MIS Quarterly (20:2) 1996, p 167. Smith, S., Winchester, D., Bunker, D., and Jamieson, R. "Circuits of Power: A Study of Mandated Compliance to an Information Systems Security De Jure Standard in a Government Organization," MIS Quarterly (34:3) 2010, p 463. Australasian Journal of Information Systems Volume 17 Number 2 2012 134 Sobreperez, P. "Using Plenary Focus Groups in Information Systems Research: More Than a Collection of Interviews," Electronic Journal of Business Research Methods (6:2) 2008, pp. 181-188. Srivastava, S., and Teo, T. "Citizen Trust Development for E-Government Adoption and Usage: Insights from Young Adults in Singapore," Communications of the Association for Information Systems (25:1) 2009, p 20. Stahl, B., Tremblay, M., and Lerouge, C. "Focus Groups and Critical Social Is Research: How the Choice of Method Can Promote Emancipation of Respondents and Researchers," European Journal of Information Systems (20:4) 2011, p 378. Straub, D. W. "Why Journals Accept Your Paper," MIS Quarterly (33:3) 2009, pp. iii-x. Strauss, A., and Corbin, J. Basics of Qualitative Research: Grounded Theory Procedure and Techniques, Thousand Oaks, California, 1990. Strauss, A., and Corbin, J. Basics of Qualitative Research, Techniques and Procedures for Developing Grounded Theory, (2nd ed.), Thousand Oaks, California, 1998. Sutton, S. G., Khazanchi, D., Hampton, C., and Arnold, V. "Risk Analysis in Extended Enterprise Environments: Identification of Critical Risk Factors in B2b E-Commerce Relationships," Journal of the Association for Information Systems (9:4) 2008, pp. 151-174. Sykes, W. "Taking Stock: Issues from the Literature on Validity and Reliability in Qualitative Research," Journal of the Market Research Society (33:1) 1991, pp. 3-12. Tams, S., and Grover, V. "The Effect of an Is Article’s Structure on Its Impact," Communications of the AIS (27) 2010, p Aricle 10. Tillotson, J., Cherry, J., and Clinton, M. "Internet Use through the University of Toronto Library: Demographics, Destinations, and Users' Reactions," Information Technology and Libraries (14:3) 1995, pp. 190-198. Torkzadeh, G., Chang, J. C.-J., and Hansen, G. W. "Identifying Issues in Customer Relationship Management at Merck-Medco," Decision Support Systems (42:2) 2006, p 1116. Tremblay, M., Hevner, A., and Berndt, D. "Focus Groups for Artifact Refinement and Evaluation in Design Research," Communications of the Association for Information Systems (26) 2010, p 83. van Velsen, L., Huijs, C., and van der Geest, T. "Eliciting User Input for Requirements on Personalization: The Case of a Dutch Erp System," International Journal of Enterprise Information Systems (4:4) 2008, p 34. Venkatesh, V., Morris, M., Davis, G., and Davis, F. "User Acceptance of Information Technology: Toward a Unified View," MIS Quarterly (27:3) 2003, pp. 425-478. Wang, P. "Chasing the Hottest It: Effects of Information Technology Fashion on Organizations," MIS Quarterly (34:1) 2010, pp. 63-85. Wang, Y., Lo, H.-P., and Yang, Y. "An Integrated Framework for Service Quality, Customer Value, Satisfaction: Evidence from China's Telecommunication Industry," Information Systems Frontiers (6:4) 2004, p 325. Weidong, and Lee "Complexity of Information Systems Development Projects: Conceptualization and Measurement Development," Journal of Management Information Systems (22:1), Summer2005 2005, pp. 45-83. Xia, W., and Lee, G. "Grasping the Complexity of Is Development Projects," Association for Computing Machinery. Communications of the ACM (47:5) 2004, p 68. Xu, M., Ong, V., Duan, Y., and Mathews, B. "Intelligent Agent Systems for Executive Information Scanning, Filtering and Interpretation: Perceptions and Challenges," Information Processing & Management (47:2) 2011, p 186. Yin, R. K. Case Study Research: Design and Methods Sage Publications, Thousand Oaks, CA, 1994. Australasian Journal of Information Systems Volume 17 Number 2 2012 135 Appendix A. Basic Guidelines for THE Focus Group Research METHOD The table below briefly summarizes guidelines for focus group research in information systems adapted from (Fern 2001; Krueger 1994; Morgan 1997), which are focused on selection of participants, how to run the sessions, the role of the moderator, and the physical setting. Substantial research has been conducted in the field of marketing analyzing the effects of variations in these guidelines (e.g. smaller or large group size, personality of the moderator, etc.). For example, it has been found that eight-member groups generate more ideas than four-member groups (Fern 1982). Category Guideline Description Participants Homogeneous strangers To achieve high levels of interactions and participation, subjects must be relatively “compatible” with each other. Job hierarchy, age, knowledge base, or other visible differences can hinder open discussions and relatively equal participation necessary to reap the benefits of focus groups. Anonymity (no demographics collected except observed) can increase participation. Knowledgeable about topic (“experts”) Since the purpose of the focus groups is to gather insights from participants, it is important that they are able to offer their views in the discussion. It is important that the expertise of the participants is relevant to the research question of interest. This is needed for validity. Three to 10 participants per group While some suggest that expert moderators can handle groups up to 15 participants, the most common number of participants in research is five to seven per group. Smaller groups can present challenges since the interactions are limited, interactions being an integral part of the focus group process. Number of sessions is function of saturation When no further insight is gained from additional data collection, saturation is reached. Moderator High involvement and discussion leadership The moderator must be highly involved in the discussion, asking questions but also probing for further insights when appropriate. Ensures everyone participates The moderator must try to facilitate productive discussions among group members. It can help to place potential “dominators” close to the moderator. Physical setting Round or rectangle tables Participants should be able to see each other and have a sense that they are all on an equal footing in the discussion. Familiar and/or relaxed setting A relaxed setting is needed for participants to become fully engaged in the conversation without being distracted by their environment. Ground rules One person speaking at a time and no side conversations The moderator must ensure that these rules are known in advance and well respected. This ensures that all discussions are captured and that all potential findings are explored. Session and protocol Topic introduced in general terms The moderator must set the stage on the topic to get everyone in the right mindset. Icebreaker or easy question first The moderator must ensure that everyone becomes quickly comfortable participating, and an icebreaker question allows him or her to bring in less vocal participants right away. Start with broad and open ended questions; then relatively structured when probing further Open-ended questions help gain a deeper understanding of respondents’ attitudes and opinions and avoid biasing participants toward specific issues researchers deem more important. Structured protocol: ask the same questions, in a similar order, across groups Protocol must be tested and modified several times after initial development. Asking the same questions allows comparisons across groups as a validation of findings, although new questions can emerge after the first few groups. Session is recorded. A scribe can take notes based on observations of the session. Researchers should have backup devices! Recorded sessions are transcribed into text files, which can then be used with qualitative analysis software such as Atlas.ti or NVivo. Video sessions offer the additional possibility of coding for non-verbal cues. One to two hours maximum Fatigue can become an issue if longer sessions are held, for both the moderator and the participants.